-- Logs begin at Tue 2022-06-07 08:30:34 UTC, end at Tue 2022-06-07 09:54:21 UTC. -- Jun 07 09:06:44.555529 dv-cinder-836996-31-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 09:06:44.564193 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 09:06:44 2022] *** Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: nodename: dv-cinder-836996-31-iscsi Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: machine: x86_64 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: clock source: unix Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: pcre jit disabled Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: detected number of CPU cores: 4 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: current working directory: / Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: detected binary path: /usr/bin/uwsgi-core Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: your processes number limit is 47680 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: your memory page size is 4096 bytes Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: detected max file descriptor number: 2048 Jun 07 09:06:44.566302 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: lock engine: pthread robust mutexes Jun 07 09:06:44.567032 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: thunder lock: enabled Jun 07 09:06:44.567032 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 09:06:44.567032 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 09:06:44.611985 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: Python main interpreter initialized at 0x55b391aee9c0 Jun 07 09:06:44.611985 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: python threads support enabled Jun 07 09:06:44.611985 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: your server socket listen backlog is limited to 100 connections Jun 07 09:06:44.611985 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: your mercy for graceful operations on workers is 90 seconds Jun 07 09:06:44.612355 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 09:06:44.612446 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: *** Operational MODE: preforking *** Jun 07 09:06:44.612745 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: *** uWSGI is running in multiple interpreter mode *** Jun 07 09:06:44.612745 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: spawned uWSGI master process (pid: 80893) Jun 07 09:06:44.612944 dv-cinder-836996-31-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 09:06:44.614245 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: spawned uWSGI worker 1 (pid: 80906, cores: 1) Jun 07 09:06:44.614245 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: spawned uWSGI worker 2 (pid: 80907, cores: 1) Jun 07 09:06:44.614245 dv-cinder-836996-31-iscsi devstack@keystone.service[80893]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 09:06:50.279862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 09:06:50.279862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: from cryptography.utils import int_from_bytes Jun 07 09:06:50.280627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 09:06:50.280627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: from cryptography.utils import int_from_bytes Jun 07 09:06:50.280860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 09:06:50.280860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: from cryptography.utils import int_from_bytes Jun 07 09:06:50.281332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 09:06:50.281332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: from cryptography.utils import int_from_bytes Jun 07 09:06:50.329683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 09:06:50.329853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 09:06:50.330040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] command line args: [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 09:06:50.330351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 09:06:50.330589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 09:06:50.330800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 09:06:50.330884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ================================================================================ {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 09:06:50.331000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] command line args: [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 09:06:50.331091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] admin_token = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 09:06:50.331352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] config_dir = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ================================================================================ {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 09:06:50.331529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] config_source = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] admin_token = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] debug = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.331936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] config_dir = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] config_source = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] debug = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.332969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_config_append = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_dir = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.333912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_file = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_config_append = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_dir = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_file = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.334919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.335944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.336958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] publish_errors = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.337868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] publish_errors = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.338987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] standard_threads = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] standard_threads = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] use_journal = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.339958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] use_json = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] use_stderr = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] use_syslog = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] use_journal = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] use_json = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.340815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.340898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] use_stderr = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.341024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] use_syslog = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.341273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 09:06:50.341505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.341976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.342922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.343933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.344898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.345867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.346946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.347985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.348930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.349924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.350952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.351961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.352999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.353921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.354924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.355982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.356953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.357958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.358887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.359956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.360914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.361917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.362958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.363942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.364638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.364736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.364888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.364976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.external = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.365929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.external = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.password = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] auth.token = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.366978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.password = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] auth.token = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.367996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.368939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.369867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.370994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.371966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.372942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.373954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.374894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.375982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.376062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.377733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.377826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.378921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.379676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.380402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.380529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.380684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.380833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.380966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.381903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.382959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.383971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.384896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.385950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.386925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.387903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.388983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.389896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.390901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.391923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.392937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.393954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.394897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.395963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.396973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.397926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.398869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] role.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.399964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] role.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.400926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] role.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] role.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.401954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.402911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.403934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.404963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.405943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.406861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.407940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.408983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.409915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.410852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.411972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.connection = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.412989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.connection = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.413927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.414917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.415855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.416894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80907) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 09:06:50.416966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.417200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.417430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.417670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.417906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 09:06:50.418077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 09:06:50.445813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.446036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.446173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.446274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.446409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.446636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.454559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.454967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.483185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.483409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.483610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.483845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.536911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.537048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.585557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.585721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.585810 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.585950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.652947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.653033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.653158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.653241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.653429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.653647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.676687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.676827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.676903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677014 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.677994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.678127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.688714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.689101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.701903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80907) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.702910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 09:06:50.719949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.720152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 09:06:50.720415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.720499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.720648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 09:06:50.720733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 09:06:50.720946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 09:06:50.721258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.721949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.722209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.733984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.734251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.734532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.742092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.742897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.745366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.745637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.745910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.746463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.746976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.747846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.748021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.748109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.748302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.748574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.748866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.775766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.776752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.777395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.777669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.778368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.778644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.783037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.783312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.783587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.784044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.784326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.784627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.790935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.791171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.791974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.792256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.795615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.796165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.796436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.796728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.796813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.797003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.797856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.798129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.798395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.798775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.816138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.816411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.818327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.818602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.821528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.823228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.825253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.825544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.827046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.827340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.830532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.832386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.834719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.836600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.838358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.838644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.838919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.839192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.839471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.839749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.840241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.840541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.840861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.841146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.841432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.841709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.858676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.858952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.859221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.859477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.860867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.861151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.861427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.861689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.867373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.869684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.870504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.870798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.872922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.873201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.873888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.874162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.874434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.874704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.874980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.876267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.876560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.876863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.877143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.877439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.889153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.889427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.889705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.889983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.890259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.890530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.890811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.891683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.891963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.892246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.892529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.892837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.893114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.893399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.909254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.911972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.912508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.915261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.915950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.917576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.918770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.919184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.919456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.919731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.920381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.922056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.922333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.922613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.927073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.928709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.930064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.931693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.932017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.932300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.932606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.932878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.935085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.935377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.935663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.935954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.941899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.942707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.942988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.945656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.945946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.946227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.950927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80907) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.951201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.951472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.951742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.952006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.952268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.952542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.952831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.953102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.953365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.953625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.953918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.954147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.954268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 09:06:50.954400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80907) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.954546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.954823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.955099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.955373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.955643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.955912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.956184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.956453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.956755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.957026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.957301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.957574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.957887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 09:06:50.992633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b391aee9c0 pid: 80907 (default app) Jun 07 09:06:50.995967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea186b4-4619-476c-ae01-226b18b502f8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:06:50.996253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea186b4-4619-476c-ae01-226b18b502f8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:06:50.996337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b391aee9c0 pid: 80906 (default app) Jun 07 09:06:50.996525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea186b4-4619-476c-ae01-226b18b502f8 None None] PATH_INFO: `/v3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:06:51.000650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Tue Jun 7 09:06:50 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:06:59.865836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:06:59.866397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:06:59.866492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:06:59.870707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:06:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:06:59.875988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:06:59.876430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:06:59.876550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:06:59.944595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: INFO dbcounter [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] Registered counter for database keystone Jun 07 09:06:59.965957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG oslo_db.sqlalchemy.engines [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80907) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 09:06:59.972034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writer thread running {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 09:07:00.253678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG passlib.handlers.bcrypt [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80907) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 09:07:00.254111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG passlib.handlers.bcrypt [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80907) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 09:07:00.310513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:00.312093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:00.443966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:06:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 09:07:00.478027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:00.546886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: INFO dbcounter [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] Registered counter for database keystone Jun 07 09:07:00.566267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG oslo_db.sqlalchemy.engines [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80906) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 09:07:00.572431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writer thread running {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 09:07:00.793530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9076-cf4f-4f5d-85e4-ac1e40854f8f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:00.797841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:00.798444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:00.798725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:00.798998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:00.807089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:00.858412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING keystone.common.rbac_enforcer.enforcer [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 09:07:00.858749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ef241-0020-4a01-88c8-dd9a24312c0d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:00.868500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Jun 7 09:07:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:07:00.874683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:00.890925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379ed4-4a77-4630-b0bb-3404bbd2fc42 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:00.895458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:00.896598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:00.896890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:00.897179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] PATH_INFO: `/v3/projects/047614873c7a462ea30132f8c49e5842` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:00.898686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] RBAC: Authorizing `identity:get_project(project_id=047614873c7a462ea30132f8c49e5842)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:00.949623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 09:07:00.949963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b6be-1b9a-498f-b65e-6fc19c493571 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:00.955766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Tue Jun 7 09:07:00 2022] GET /identity/v3/projects/047614873c7a462ea30132f8c49e5842 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:03.172482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdb9a-d85c-4a16-ba5c-bde69e0cef04 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:03.173025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdb9a-d85c-4a16-ba5c-bde69e0cef04 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:03.173127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdb9a-d85c-4a16-ba5c-bde69e0cef04 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:03.177685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:03.183107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:03.183485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:03.183666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:03.202584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:03.204108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:03.211946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 07 09:07:03.234133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdb9a-d85c-4a16-ba5c-bde69e0cef04 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:03.246863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdb9a-d85c-4a16-ba5c-bde69e0cef04 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:03.251031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:03.251650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:03.251937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:03.252213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:03.260083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:03.261820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe38d4d-b983-4667-9f9e-b1da9a70eff7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:03.271382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Jun 7 09:07:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:07:03.276427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:03.285348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a120-a2c9-4318-9b20-87a639e51007 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:03.289761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:03.290608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:03.290888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:03.291160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:03.292533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:03.294397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601a1ef-dcbe-492c-950e-4a12d49e9e8b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:03.295939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Tue Jun 7 09:07:03 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:05.560378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ca566-6adc-4137-bbcc-c7093322ae6a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.560939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ca566-6adc-4137-bbcc-c7093322ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.561023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ca566-6adc-4137-bbcc-c7093322ae6a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.562334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:05.568256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797734be-d7a1-426e-8803-653fe223c647 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.568679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797734be-d7a1-426e-8803-653fe223c647 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.568849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797734be-d7a1-426e-8803-653fe223c647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.585176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-797734be-d7a1-426e-8803-653fe223c647 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:05.586346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-797734be-d7a1-426e-8803-653fe223c647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:05.626065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4552c-cd5b-4b65-b2ef-2b7b476508c5 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.626355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4552c-cd5b-4b65-b2ef-2b7b476508c5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.626639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4552c-cd5b-4b65-b2ef-2b7b476508c5 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.628000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:05.633714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.634008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.634291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.666243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:05.673833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG passlib.handlers.bcrypt [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80906) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 09:07:05.674266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG passlib.handlers.bcrypt [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80906) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 09:07:05.689611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797734be-d7a1-426e-8803-653fe223c647 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:05.713844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:05.722215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:05.724333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:05.793314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797734be-d7a1-426e-8803-653fe223c647 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:05.824995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:05.826665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:05.827464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.827745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.828021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.839101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:05.847173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:05.851076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:05.851953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:05.852877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:05.857930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:05.939987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:05.948521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d660-6bd8-496d-a91c-a6ed5b11f580 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:05.951022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:05.957703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3ea91-34c6-46d7-ad5d-d176b1d177b4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:05.986198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:05.987116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.987402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.987676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.992114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:05.992803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:05.993091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:05.993371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:05.994161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:05.994456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:05.995841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.009160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:06.014892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:06.059004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.060649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.114693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0c7e-1d21-490e-b556-4b06661a87aa None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:06.149027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:06.149890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:06.150172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:06.150444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:06.151645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.153132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.154743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:06 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:06.157120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:07:06.159952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:06.162761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:06.165473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:06.175768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:06.259466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426a95-e4bd-4bc8-bbab-2acb2964d93d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:06.261053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c698c-701b-42bb-aa7f-b52a68e29681 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:06.292763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:06.293550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:06.293835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:06.294011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:06.294157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:06.294813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:06.295100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:06.295268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.295379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:06.296771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.298382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:06.304640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:06.314599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.316077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:07:06.337460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:06.338448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:06.339229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:07:06.339229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:07:06.339229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:07:06.339229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:06.339229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:06.344378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:06.385143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:06.400680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:06.403434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e821-4969-41d5-8780-6d4f6c273f17 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:06.437459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:06.438313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:06.438599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:06.438874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:06.439707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.domain.id=default, user.id=a90171f1014d4a6dbe696da11bc1dd7a) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:06.440020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.441507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4e227-a1fe-46cd-97c5-c5c94cd74ecc None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.444932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5391-15bb-41e6-8c23-414b45371c9e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:06.448319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Jun 7 09:07:06 2022] GET /identity/v3/role_assignments?user.id=a90171f1014d4a6dbe696da11bc1dd7a&scope.domain.id=default&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:06.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:06.482981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:06.483266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:06.483545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:06.484296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:06.484641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:06.486071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aacb40-c33b-42f6-874f-5993181d09e7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:06.505658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:09.114249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a509f-6192-4399-b994-f9703cedbbb7 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.115239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a509f-6192-4399-b994-f9703cedbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.115735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a509f-6192-4399-b994-f9703cedbbb7 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.117409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:09.122938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fb8c-b5b1-4623-8170-d705bae6e5e4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.123761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fb8c-b5b1-4623-8170-d705bae6e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.124256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fb8c-b5b1-4623-8170-d705bae6e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.147367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-38a4fb8c-b5b1-4623-8170-d705bae6e5e4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:09.148945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-38a4fb8c-b5b1-4623-8170-d705bae6e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:09.179909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957644-c140-494b-adb1-caeb5cab9147 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.180570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957644-c140-494b-adb1-caeb5cab9147 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.181092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957644-c140-494b-adb1-caeb5cab9147 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.182739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 13/23] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:09.188727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.189468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.189967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.213699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:09.215696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:09.283636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 11/24] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:09.285894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0916b-d021-4059-af62-0519ff7643c6 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.286393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0916b-d021-4059-af62-0519ff7643c6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.286870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0916b-d021-4059-af62-0519ff7643c6 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.288546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 12/25] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:09.293839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.294381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.294858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.318553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 14/26] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:09.321298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:09.322749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:09.325644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:09.398649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:09.414724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:09.443009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2fad1-bf2e-437d-8c4b-4b9b9dde35b7 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:09.444549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 13/27] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:09.445680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:09.489235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:09.490041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.490323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.490596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.509701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:09.526611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:09.527535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 15/28] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:09.532197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:09.595061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e1fe2-c1b7-470f-ade9-f4ba276c2c58 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:09.658329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:09.659428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.659919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.660947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.661936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:09.663724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:09.714339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410506fd-550b-4b12-a6eb-ccd782544451 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:09.753046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:09.754804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:09.755289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:09.755789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:09.758064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:09.769373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:09.771881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:07:09.794365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:09.795233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:09.796711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:07:09.796711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:07:09.796711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:07:09.796711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:09.796711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 16/29] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:09.797010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:09.909498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 14/30] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Jun 7 09:07:09 2022] POST /identity/v3/roles => generated 219 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:09.911061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:09.975413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98319a-618b-4dd8-ac64-f9bd113244a2 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:09.996985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:10.080818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117b5e5-3cbd-4a4c-bd4a-1b8b7e60d1c8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.113895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.114589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.115115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.115115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.115881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:10.116192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.117644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac37d14-cb22-4565-b81c-cacc43b2363a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.123393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.124076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.124365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.124676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.125432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:10.126338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.127894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4e02-224c-423f-b2ef-ffc5a8550459 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.140729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:10.142105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10cd6-fc82-4ee7-9276-6c4aef9d4a7c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.142395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10cd6-fc82-4ee7-9276-6c4aef9d4a7c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.142673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10cd6-fc82-4ee7-9276-6c4aef9d4a7c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.143910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 18/32] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:10.147806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 15/33] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:10.149769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10cd6-fc82-4ee7-9276-6c4aef9d4a7c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:10.150972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.151415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.151701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.172780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:10.174403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:10.223909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:10.268622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 16/34] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:10.269410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:10.296018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10cd6-fc82-4ee7-9276-6c4aef9d4a7c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.300806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:10.341261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.342386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.342820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.343246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.345334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.347196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.353897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 19/35] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:10 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:10.354952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:10.394277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf58eed-b5a8-4126-ba56-7cc0f9675a28 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.434824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.435423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.435771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.436190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.437165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.438662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.478744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115c24-186f-480c-a6ad-1edc65473f2c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.496263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:10.515305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.515989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.516275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.516553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.517355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.518808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c257ce3-042f-4fff-8276-9dbe554105ea None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.553249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 17/36] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/projects => generated 301 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:10.554463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:10.654282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:10.684632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43898b2d-88d1-4a7d-9b67-ca3f4d93d097 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.725388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.726100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.727583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.727897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.729196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.730741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.732808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 20/38] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/roles => generated 225 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:10.734701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e2340-0316-4517-9651-b05b578edc66 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.735217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e2340-0316-4517-9651-b05b578edc66 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.735701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e2340-0316-4517-9651-b05b578edc66 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.736710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 18/38] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:10.737817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 21/39] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:10.741725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:10.749905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec09bf-6351-4912-aee8-6f38065efeaa None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.750566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec09bf-6351-4912-aee8-6f38065efeaa None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.751073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec09bf-6351-4912-aee8-6f38065efeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.778947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f4ec09bf-6351-4912-aee8-6f38065efeaa None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:10.780289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f4ec09bf-6351-4912-aee8-6f38065efeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:10.870387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 22/40] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:10.877792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85164-f7ad-4fa2-8151-418b2b11565b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.878258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85164-f7ad-4fa2-8151-418b2b11565b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.880174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85164-f7ad-4fa2-8151-418b2b11565b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.881232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 23/41] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:10.887369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.887957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.888229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.906855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1458db-79a9-4540-89a4-94eb666dfb0f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:10.908937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:10.910222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:10.950475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:10.951405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:10.951698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:10.951981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] PATH_INFO: `/v3/domains/default/users/a90171f1014d4a6dbe696da11bc1dd7a/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:10.954558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a90171f1014d4a6dbe696da11bc1dd7a, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:10.957011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:10.990391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 24/42] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:10.991258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:11.033408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:11.064312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 19/43] 10.222.0.57 () {58 vars in 1361 bytes} [Tue Jun 7 09:07:10 2022] PUT /identity/v3/domains/default/users/a90171f1014d4a6dbe696da11bc1dd7a/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:11.065261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:11.075117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:11.103613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1e16-52a4-488e-b37c-c2510a6617ec None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:11.140655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:11.141338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:11.141645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:11.141929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:11.142692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:11.144271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431698f-1951-46bd-af81-089c3f308467 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:11.176786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e163b2-d6b5-487c-a7e9-dfcb7ba65a2c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:11.213994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:11.214806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:11.215091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:11.215381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:11.216647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:11.218218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:11.220080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 20/44] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:11.225689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:11.236828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 25/45] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Jun 7 09:07:10 2022] POST /identity/v3/roles => generated 223 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:11.318078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:11.344755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9c93e-d7ca-49cc-8b8b-b1a6316bedb3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:11.382614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:11.383375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:11.383665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:11.383944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:11.384711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:11.386237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607cd81-32b8-43ed-9e68-8a2ae42173fc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:11.466233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 21/46] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:07:11 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:07:13.306004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1e11-2aa5-4763-9208-5f6e75918dfe None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.306531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1e11-2aa5-4763-9208-5f6e75918dfe None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.306531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1e11-2aa5-4763-9208-5f6e75918dfe None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.307851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 26/47] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:13.313116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.313573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.313715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.330827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:13.332669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:13.403404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 22/48] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:13.426431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1e11-2aa5-4763-9208-5f6e75918dfe None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.520606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:13.527641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1e11-2aa5-4763-9208-5f6e75918dfe None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:13.560902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:13.561620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.561905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.562179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.572501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-49e1dc09-10b1-45ee-907e-6686ec5fdd3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:13.580226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:13.581109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:13.581860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 27/49] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:13.587898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.643083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:13.675228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485daa7c-5bd1-4335-97f5-b34b0af7b7a1 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.675517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485daa7c-5bd1-4335-97f5-b34b0af7b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.676331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485daa7c-5bd1-4335-97f5-b34b0af7b7a1 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.677742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 28/50] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:13.682677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.682965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.683243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.686902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6aa47-668c-494e-b525-c221f6bb31c9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:13.700312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:13.701540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:13.721068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:13.721731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.722009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.722281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.723102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:13.723407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:13.724827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:13.735862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 23/51] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:13.741798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.768539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 29/52] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:13.792019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.841079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a986ac-1942-4faf-b52e-02f8ddf24bc1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:13.857843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:13.875349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:13.875520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:13.876230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.876513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.876816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.878029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:13.880033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:13.881701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 24/53] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:13 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:13.886872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.891301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a786b-5c02-444c-a8df-79ab31087937 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:13.926452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:13.927245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:13.927533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:13.927811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:13.929037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:13.930515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:13.932287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 30/54] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:13.937537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:13.985637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec9437-3047-438a-9d75-a83844c53627 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:14.024298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:14.025138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:14.025427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:14.025708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:14.026969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:14.028504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:14.030263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 25/55] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:14.035863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:14.037091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec67e-2fca-49de-be50-e3e11939e90e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:14.071994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:14.072798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:14.073159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:14.073508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:14.074289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:14.075828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82cd77-2c2a-4849-8d7c-1873bdd7636c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:14.091033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:14.134340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f424ca-333a-47f6-8be7-3f14a36ba3bf None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:14.166168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:14.166814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:14.167099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:14.167376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:14.168195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.domain.id=default, user.id=a90171f1014d4a6dbe696da11bc1dd7a) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:14.168505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:14.169960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d3f96-84b0-4ba4-b3c8-e4c4e1ea902a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:14.178247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 26/56] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Jun 7 09:07:14 2022] GET /identity/v3/role_assignments?user.id=a90171f1014d4a6dbe696da11bc1dd7a&scope.domain.id=default&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:14.214947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 31/57] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Jun 7 09:07:13 2022] POST /identity/v3/projects => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:07:16.474870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1fa7-6801-4c8f-9199-d651bff2c128 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:16.475371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1fa7-6801-4c8f-9199-d651bff2c128 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:16.475457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1fa7-6801-4c8f-9199-d651bff2c128 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:16.476824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 27/58] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:16.482066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:16.482468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:16.482641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:16.494075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:16.499386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:16.500622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:16.568231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 32/59] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:16.592856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1fa7-6801-4c8f-9199-d651bff2c128 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:16.692979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1fa7-6801-4c8f-9199-d651bff2c128 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:16.709173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:16.725098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:16.725865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:16.726157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:16.726438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:16.727613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:16.729115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a38b47-a749-40b5-8e78-a25ea586a885 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:16.730716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 28/60] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:16.736185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:16.833154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fecdc3-084c-4d11-92fa-b8cf6e7e69d1 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:16.865743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:16.866402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:16.866692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:16.866973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:16.867690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:16.869200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1835140-a6e1-487e-8c81-ac8e16d83a78 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:16.880173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:16.943356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 33/61] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:16 2022] POST /identity/v3/users => generated 275 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:20.250042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752a8c-5655-477a-beb7-f627d8504b0f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.251182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752a8c-5655-477a-beb7-f627d8504b0f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.251673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752a8c-5655-477a-beb7-f627d8504b0f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.254259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 29/62] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:20.259875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c64b47-b31f-46ce-9ea1-978e0d58e4e1 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.260851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c64b47-b31f-46ce-9ea1-978e0d58e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.261358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c64b47-b31f-46ce-9ea1-978e0d58e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.286115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-71c64b47-b31f-46ce-9ea1-978e0d58e4e1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:20.287664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-71c64b47-b31f-46ce-9ea1-978e0d58e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:20.381212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 34/63] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:20.416653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee752a8c-5655-477a-beb7-f627d8504b0f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:20.543977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee752a8c-5655-477a-beb7-f627d8504b0f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:20.560363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2911-e812-44bb-a1d9-2a75b24923df None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.560693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2911-e812-44bb-a1d9-2a75b24923df None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.560959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2911-e812-44bb-a1d9-2a75b24923df None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.562493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 35/64] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:20.574442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb4ab5-4451-45e7-bf31-cd91ebc67edc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.574744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb4ab5-4451-45e7-bf31-cd91ebc67edc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.575008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb4ab5-4451-45e7-bf31-cd91ebc67edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.595276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4adb4ab5-4451-45e7-bf31-cd91ebc67edc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:20.596494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4adb4ab5-4451-45e7-bf31-cd91ebc67edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:20.608730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:20.609887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.610373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.610845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.623913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-5c4c7dda-acc4-4591-af07-b5fc10a373a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:20.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:20.634189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:20.634932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:20.637300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 30/65] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:20.639013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb7bb-e4be-482c-a03e-e635affb064a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.639518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb7bb-e4be-482c-a03e-e635affb064a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.639990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb7bb-e4be-482c-a03e-e635affb064a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.641490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 31/66] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:20.642884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fb7bb-e4be-482c-a03e-e635affb064a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:20.691025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 36/67] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:20.692352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.692676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.692965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.704452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:20.724755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:20.726458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:20.770637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fb7bb-e4be-482c-a03e-e635affb064a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:20.787151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:20.807147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:20.807459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 37/68] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:20.812983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.812983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.813140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:20.813205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.813205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:20.813205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:20.813205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3edb-3421-49f1-984e-4d24071be33f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:20.824787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 32/69] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:20.826112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9dd4-64de-4cfc-a2ff-9ddac813b96e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.826407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9dd4-64de-4cfc-a2ff-9ddac813b96e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.826686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9dd4-64de-4cfc-a2ff-9ddac813b96e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.827933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 33/70] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:20.834074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.834504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.834859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.854739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:20.855920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:20.918634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd133f9-6dd0-4333-b031-0d4687849dde None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:20.929100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 34/71] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:20.930160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:20.964656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:20.965458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:20.965740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:20.966016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:20.973754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:20.976770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:20.984309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:20.989009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 38/72] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:20.989781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.037793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.049536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893df3d-7493-4a5b-84e6-66917d9abcb8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.086131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.086926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.087208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.087482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.098580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2fd5d-3095-43f7-aecc-7678d3f35f87 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.106735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.108243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.110012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 35/73] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:20 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.113998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.132316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.133132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.133419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.133702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.143862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:21.151464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:21.152281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:21.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 39/74] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:21.153677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.213377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.218977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f01b0-06a2-4090-942b-41d18a72c414 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.251203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cab7-8e8b-41ab-bbc4-69878759d67a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.253021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.253677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.254486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.254773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.255055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.265652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-4ce20135-fec9-49fd-962e-7990181c36de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:21.274297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:21.275121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:21.275886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 36/75] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:21.276725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1078fd-308e-4fb3-bcbe-927b864aff4e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.277017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1078fd-308e-4fb3-bcbe-927b864aff4e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.277299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1078fd-308e-4fb3-bcbe-927b864aff4e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.278560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 37/76] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:21.279259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1078fd-308e-4fb3-bcbe-927b864aff4e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.288854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.289532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.289822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.290102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.290854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:21.291163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.292572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.301849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.302812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.387675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1078fd-308e-4fb3-bcbe-927b864aff4e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.400688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085abd02-fdd8-44a2-a126-f4330366bcdc None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.420614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.421277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.421557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.421841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.433524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.434186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.434479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.434757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.435504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:21.435813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.437240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.445344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 41/78] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.446124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.451024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.472774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.494372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.495857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.542789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea376-0063-4e2f-859f-eb1dba9bd31d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.577464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.578367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.578656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.578940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.579692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:21.580004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.581433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17704e5-34a0-4ca6-b8fe-92f7e475155c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.589802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 42/79] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.590968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.591263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.591547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.600504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 38/80] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:21.601310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.610144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:21.611359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:21.655616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.678417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 43/81] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:21.679078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.699441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62517a7-2579-40f6-8281-2248c1fb4083 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.702917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.734148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.735031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.735319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.735598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.736826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.738367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.740003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 39/82] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.740807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.788487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49937f-2e24-44a2-afad-20df838cf909 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.821766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.822617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.822896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.823167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.824358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.825836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.827607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 44/83] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.828272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.845472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3de22-f2fe-4f4e-9a6a-7452eca71a9e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.868701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.879615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.880478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.880789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.883110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.883110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.883687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.885389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 40/84] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.886138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.911256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:21.926885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466fe6c-1df9-4ada-9b45-bba205a3828d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:21.960826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:21.961674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:21.961955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:21.962230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:21.963414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:21.964880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:21.966562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 45/85] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:21.967392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:21.989571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e38cb-fc73-4365-bc92-7ddbb851453e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.025681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.026472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.026764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.027047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.028241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.029743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.031370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 41/86] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:22.032136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.065341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc423153-7f43-491e-8136-bc968c971c0a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.090596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.099350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.100000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.100288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.102929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.124733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.132414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6f236-764f-4355-8f05-306d942eddd5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.166382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.167030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.167318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.167595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.171098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.172565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.234686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.236151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.268767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 46/87] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:22.269674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.294565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.325175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.335712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 42/88] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:22.336550 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.369375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eef72-dbda-428e-84a0-c2622aa0aa5f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.402519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.403214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.403502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.403775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.435795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268c87-ac93-420a-b1ac-228210ab7f65 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.468967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.469372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.470026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.470311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.470440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.470591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.471433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:22.471753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.473738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.479169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 43/89] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:22.479169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.520965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.530332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.580938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 47/90] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:22.582639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.610739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825dea4-5800-480c-88c5-bf2421c669b3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.648859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.649585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.649834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.650113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.650853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.652332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.690768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb54e16-22d6-4811-bcce-123fb5df82e6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.721228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 44/91] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:22 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:22.728158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.729132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.729493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.729780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.730323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.731261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.732928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.736352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 48/92] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:22.737471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.775103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.828952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:22.849816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14dd3-9ced-4a83-a27e-b1d60410301e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.891494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e0b40-7daa-476b-919d-4b111387dec8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:22.892255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.893289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.893587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.893859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] PATH_INFO: `/v3/projects/2fa55a4eeb5545c09984059721a29075` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.895724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] RBAC: Authorizing `identity:get_project(project_id=2fa55a4eeb5545c09984059721a29075)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.897266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.899163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 45/93] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/projects/2fa55a4eeb5545c09984059721a29075 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:22.904813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:22.937174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:22.938245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:22.938546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:22.938884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:22.940862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:22.942510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:22.945345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 49/94] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:22.946171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:23.029466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee0da-c32d-4943-8eec-9f1ee14031a9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:23.071151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:23.078444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=a90171f1014d4a6dbe696da11bc1dd7a) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:23.085616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:23.086037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f37b-8ee4-487e-9def-974dcdce0272 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:23.093281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 46/95] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/role_assignments?user.id=a90171f1014d4a6dbe696da11bc1dd7a&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:23.094405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:23.117317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:23.130454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=2fa55a4eeb5545c09984059721a29075, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:23.143133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a709090-1a7c-4a20-b3f6-3be8f22bbede None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:23.143643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 50/96] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:22 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=2fa55a4eeb5545c09984059721a29075&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:23.224469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5b46-c0d9-4a37-82ca-672a45d715c1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:23.272282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:23.273010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:23.273299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:23.273579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:23.274448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:23.274768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:23.276281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e0cd-bbcf-4dbf-ac93-e90cf082f72e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:23.283563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 47/97] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:23 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:25.302236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440afc7-3f8f-42b7-8a6a-6d22be648521 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:25.303235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440afc7-3f8f-42b7-8a6a-6d22be648521 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:25.303734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440afc7-3f8f-42b7-8a6a-6d22be648521 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:25.305722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 51/98] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:25.311467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:25.312304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:25.312833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:25.337596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:25.339174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:25.430875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 48/99] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:25.478124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440afc7-3f8f-42b7-8a6a-6d22be648521 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:25.620070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440afc7-3f8f-42b7-8a6a-6d22be648521 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:25.666974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:25.667782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:25.668071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:25.668349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:25.685478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:25.693944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:25.694815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:25.696006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 52/100] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:25.704632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:25.720548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:25.837872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd15a26-d44f-4822-9ffe-6bbf4e6901f7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:25.887533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:25.888605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:25.889107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:25.889597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:25.890551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:25.891060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:25.892675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d586-63a5-4615-bea0-761ea2e29a93 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:25.904962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 49/101] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:25.911041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:25.940490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:26.158703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f1c7-4bb2-43a6-91d4-97611075b694 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.159391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f1c7-4bb2-43a6-91d4-97611075b694 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.160500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f1c7-4bb2-43a6-91d4-97611075b694 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.162764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 50/102] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:26.169261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.169881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.170369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.186695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13e01-cf1a-4d6c-b3d0-02b9db3ac9a7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:26.205687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:26.207272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:26.246954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:26.247850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.248139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.248419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.249830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:26.251639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1a1bf-72a1-450c-b37a-fae68d5ffcb9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:26.254174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 53/103] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:25 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:26.256264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13b3ae-e0b1-4e2b-a56c-22a6aae47e20 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.256554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13b3ae-e0b1-4e2b-a56c-22a6aae47e20 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.256860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13b3ae-e0b1-4e2b-a56c-22a6aae47e20 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.258547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 54/104] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:26.265650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.265946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.266231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.286362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:26.288172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:26.351595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:26.361755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 51/105] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:26.363691 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.424841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 55/106] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:26.427137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.449909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:26.534320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf014-bf52-4775-acd5-b3051512e902 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:26.555753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb4b64-0ca1-486a-99f0-2823c66d4455 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:26.593434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:26.594668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.595173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.595703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.605960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:26.606842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.607134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.607408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.608711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:26.609452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:26.610672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:26.612671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 52/107] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:26 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a51e31db-d87d-48ab-8445-522a93ce8c7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:26.617485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:26.618314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:26.619432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 56/108] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:26.620076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76428-9ea1-40bb-86c3-14b75d26487c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.620346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76428-9ea1-40bb-86c3-14b75d26487c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.620500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76428-9ea1-40bb-86c3-14b75d26487c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.620765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.622492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 57/109] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:26.626025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76428-9ea1-40bb-86c3-14b75d26487c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.709783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:26.716457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:26.742198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76428-9ea1-40bb-86c3-14b75d26487c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:26.760085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f4de-4e94-4b8d-b660-02188f915160 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:26.779599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:26.780485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.780801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.781078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/users/9d949c21d8c14c66b498639dd44ff225/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.783079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, user_id=9d949c21d8c14c66b498639dd44ff225, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:26.785096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:26.796095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:26.796926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.797238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.801767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.808689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-4520b804-35cb-424f-9a4e-9bac112c444f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:26.816569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:26.817603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:26.818576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 53/110] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:26.819445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10981ed6-9167-4355-8bbb-0be5e892ed06 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.819445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10981ed6-9167-4355-8bbb-0be5e892ed06 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.819604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10981ed6-9167-4355-8bbb-0be5e892ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.835800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-10981ed6-9167-4355-8bbb-0be5e892ed06 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:26.837014 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-10981ed6-9167-4355-8bbb-0be5e892ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:26.881237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 58/111] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:26 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/users/9d949c21d8c14c66b498639dd44ff225/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:26.882277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.910736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 54/112] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:26.920875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88876c-cf4f-4d9f-99f5-6427b5274085 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.920875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88876c-cf4f-4d9f-99f5-6427b5274085 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.920875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88876c-cf4f-4d9f-99f5-6427b5274085 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.922156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 55/113] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:26.923005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449216b-6743-4e7a-a166-72d299087590 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:26.924076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449216b-6743-4e7a-a166-72d299087590 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:26.924076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449216b-6743-4e7a-a166-72d299087590 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:26.925975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 56/114] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:26.926911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449216b-6743-4e7a-a166-72d299087590 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:26.999164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.004575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc0cfb-335d-4381-b470-7a900c60ffa0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.040403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.045496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.047968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.047968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.047968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.047968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:27.047968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.049722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bfbcc-5f86-4555-b998-ac5eafd75264 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.053452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449216b-6743-4e7a-a166-72d299087590 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.060721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 59/115] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.067322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.067629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.068313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.086752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:27.087960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:27.097759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.098516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.098728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.099009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.099771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:27.100078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.101491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46cdb5-6089-4f0d-aab3-ddf786e21ddd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.110276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 57/116] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.113553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.113553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.113553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.129763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:27.131194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:27.164475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 60/117] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:27.172741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.204438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 58/118] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:27.205180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.262591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.295417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e806-813a-455b-8fd4-4a107ecf8502 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.317349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7e59-b036-494b-86fd-b652c411f92b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.323306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.337660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.338466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.338757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.339048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.350540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:27.353930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.354804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.355090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.355369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.356567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.358072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:27.358297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:27.359128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:27.359874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 61/119] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:27.360488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 59/120] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.360558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.360659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.467003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6b2c3-d5ed-42ec-bbd9-8cfabcbfee9e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.480414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ad24-9d4c-4538-8170-c8bb0df57f78 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.483983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.503729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.504903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.505401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.505734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.517128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:27.517241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.517997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.518283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.518559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] PATH_INFO: `/v3/users/d5275bfb53864285ae97a884e77aef1b` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:27.525298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:27.526180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:27.527284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 60/121] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:27.530133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.535826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] RBAC: Authorizing `identity:get_user(user_id=d5275bfb53864285ae97a884e77aef1b)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.537346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.543666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 62/122] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/users/d5275bfb53864285ae97a884e77aef1b => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.543666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.587802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.641555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e734a-82b5-466e-8aa7-36b450c38e42 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.648468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b15037-641d-4628-96cd-5fbd34d2e94f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.677064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.677938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.678233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.678516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.679726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.681210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.682843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 61/123] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.683609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.686096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.686964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.687343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.687859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.689031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.690494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.696908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 63/124] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.698415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.733162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.795447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36c280-acab-49b2-ad35-bf5f913043e7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.805333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa057f3-ad6a-4bdf-9193-a204e3b6bdc3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.817272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.830972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.831840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.832087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.832400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.833195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:27.833508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.834931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.840620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.841363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.841729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.842083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:27.842911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:27.843318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:27.843767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 62/125] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.844977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:27.849133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.859030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 64/126] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:27.859030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:27.959615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751cc80-dd09-4b30-9f8c-7dc02b3031d1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.964726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05834b4-387b-49f4-a6b6-e02774b8794f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:27.968660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:27.996463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37513cff-8576-427f-bc30-22062d146723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:27.997304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513cff-8576-427f-bc30-22062d146723 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:27.997627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513cff-8576-427f-bc30-22062d146723 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:27.997981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513cff-8576-427f-bc30-22062d146723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.000298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.001108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.001462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.001819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.002668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.004456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.074283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37513cff-8576-427f-bc30-22062d146723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.075665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37513cff-8576-427f-bc30-22062d146723 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.089101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 65/127] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:07:27 2022] POST /identity/v3/projects => generated 302 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:28.092721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.124561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.197193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 63/128] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:28.198360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37513cff-8576-427f-bc30-22062d146723 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.209639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.248436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b148c5-a1db-4c84-ae64-65a781674e21 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.289023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.290323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.290795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.291300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/users/a90171f1014d4a6dbe696da11bc1dd7a/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.293690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, user_id=a90171f1014d4a6dbe696da11bc1dd7a, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.295436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.328003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37513cff-8576-427f-bc30-22062d146723 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.366444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.367319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.367606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.367886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.369145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.370686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.372457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 64/129] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.373341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.383066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 66/130] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:28 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/users/a90171f1014d4a6dbe696da11bc1dd7a/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:28.385819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.466069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.494448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1b14-20ad-4197-8436-9cfccb2eabae None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.498366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.519325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddd62-0534-47b1-aae5-8ec07b688e1c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.532827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.533725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.534194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.534315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.536097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.537600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 65/131] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.540111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.564251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.565517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.566067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.566557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.568269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.570128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.573197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 67/132] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.574577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.653244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9a40b-e18c-4f34-a8b5-725186961b00 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.691047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be1f1f-31cc-4555-b7c2-a26e24baab03 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.698050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.698919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.699201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.699477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] PATH_INFO: `/v3/projects/2fa55a4eeb5545c09984059721a29075` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.700714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] RBAC: Authorizing `identity:get_project(project_id=2fa55a4eeb5545c09984059721a29075)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.702771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.705018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 66/133] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/projects/2fa55a4eeb5545c09984059721a29075 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.705794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.724321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.740614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.740614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.740614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.740614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.742749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.744270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.749637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 68/134] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.750420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.784414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:28.893091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c9f4-6ea7-4b2b-9bc2-9f7d7b9d4bec None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:28.931258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:28.931921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:28.932205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:28.932484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:28.933370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=d5275bfb53864285ae97a884e77aef1b) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:28.933684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:28.935408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:28.942507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 67/135] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:28 2022] GET /identity/v3/role_assignments?user.id=d5275bfb53864285ae97a884e77aef1b&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:28.943306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:28.944186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f4eb1-eaa5-4f70-aad5-d4b54909a415 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:29.048409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:29.076907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:29.077985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:29.078344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:29.078700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] PATH_INFO: `/v3/projects/2fa55a4eeb5545c09984059721a29075/users/9d949c21d8c14c66b498639dd44ff225/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:29.080948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fa55a4eeb5545c09984059721a29075, user_id=9d949c21d8c14c66b498639dd44ff225, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:29.082286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9917541-b099-4312-a12c-f1c19cbe9034 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:29.083124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7963c-3668-4d7c-b55f-3ed547629a73 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:29.133458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:29.134376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:29.134668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:29.134946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/users/9d949c21d8c14c66b498639dd44ff225/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:29.137531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, user_id=9d949c21d8c14c66b498639dd44ff225, role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:29.138983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25173601-23ef-4ee6-8cea-9f2b32364547 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:29.166548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 69/136] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:28 2022] PUT /identity/v3/projects/2fa55a4eeb5545c09984059721a29075/users/9d949c21d8c14c66b498639dd44ff225/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:29.197641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 68/137] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:28 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/users/9d949c21d8c14c66b498639dd44ff225/roles/2c535dd04ec64d44949ee9193f6d696e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:29.383756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ea4e-3b09-4505-96ed-2c07cd424aeb None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:29.384155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ea4e-3b09-4505-96ed-2c07cd424aeb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:29.384331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ea4e-3b09-4505-96ed-2c07cd424aeb None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:29.386091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 70/138] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:29.392224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:29.392763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:29.392834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:29.415679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:29.418300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:29.580171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 69/139] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:29.622072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ea4e-3b09-4505-96ed-2c07cd424aeb None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:29.773026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:29.796686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ea4e-3b09-4505-96ed-2c07cd424aeb None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:29.853789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:29.854521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:29.854806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:29.855085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:29.873056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:29.887058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:29.888197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:29.889063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:29.889650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 71/140] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:29.895025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:30.031769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:30.082468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e61e-12b8-4ae4-befd-9f0762ad7b16 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:30.130276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:30.130871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:30.131155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:30.131428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:30.132179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:30.132484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:30.133901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:30.148945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 70/141] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:30.154218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:30.280651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04976f9f-2749-4b51-859e-2b783034d809 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:30.322701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:30.323586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:30.323873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:30.324159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:30.325418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:30.326876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:30.328710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 72/142] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:30 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:30.334736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:30.471526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944876-0558-4c16-8920-29d7eb695a12 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:30.498372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:30.508043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:30.508729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:30.509040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:30.509322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:30.583204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:30.584710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:30.717068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 71/143] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:30.723372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:30.744962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:30.865688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3619b-de4e-46d8-ad39-a08f23f75b8d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:30.918274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:30.919574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:30.920056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:30.920549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:30.922004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:30.923696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc66ad4-6331-4519-9466-6f592c197018 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:30.926210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:30 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:30.932534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:30.975371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:31.094731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c6d50-8232-46d9-87ef-a0faa7734491 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:31.134089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:31.134756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:31.135039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:31.135318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:31.136159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:31.136472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:31.138202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd595b0b-35c6-4930-b2da-d7bb8991ff27 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:31.144819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:30 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:31.434653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6f65-3c2b-4e9c-bc0e-d546db0de9ec None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:31.434653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6f65-3c2b-4e9c-bc0e-d546db0de9ec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:31.434653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6f65-3c2b-4e9c-bc0e-d546db0de9ec None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:31.434653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 74/146] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:31.441058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:31.441443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:31.441820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:31.472383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:31.473671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:31.549716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 73/147] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:31.575436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b6f65-3c2b-4e9c-bc0e-d546db0de9ec None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:31.659676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:31.684848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b6f65-3c2b-4e9c-bc0e-d546db0de9ec None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:31.720688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:31.721498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:31.721854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:31.722136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:31.723330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:31.724845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c040f1-6dd6-436d-9298-c4286936a373 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:31.726486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 75/148] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:31.731977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:31.754272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:31.848449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa022d9-4d95-4dea-aab5-e214d6cec2c4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:31.889875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:31.890561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:31.890846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:31.891116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:31.891843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:31.893323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ca4af-e282-424a-a117-3f595064252f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:32.002685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 74/149] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:31 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:32.335807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90141f48-d129-4b7b-92e0-13ec2331e0d1 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.336145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90141f48-d129-4b7b-92e0-13ec2331e0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.336329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90141f48-d129-4b7b-92e0-13ec2331e0d1 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.337708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 76/150] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:32.343333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd81e5-f1ff-4694-92f3-bc0d863d6be4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.343619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd81e5-f1ff-4694-92f3-bc0d863d6be4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.343891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd81e5-f1ff-4694-92f3-bc0d863d6be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.360606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38293b6a-1b8a-485c-a936-04585b58150f None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.361212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38293b6a-1b8a-485c-a936-04585b58150f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.361688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38293b6a-1b8a-485c-a936-04585b58150f None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.363799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 77/151] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:32.366702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3edd81e5-f1ff-4694-92f3-bc0d863d6be4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:32.368985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.369482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.369978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.374553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3edd81e5-f1ff-4694-92f3-bc0d863d6be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:32.388519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:32.390052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:32.473360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 75/152] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:32.478175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 78/153] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:32.503479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de279-8c2b-43a0-a8fb-181b504d834d None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.503765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de279-8c2b-43a0-a8fb-181b504d834d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.504209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de279-8c2b-43a0-a8fb-181b504d834d None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.506347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 76/154] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:32.506994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:32.511628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.511920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.512501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.528959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:32.531492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:32.533496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:32.588340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:32.621929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 77/155] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:32.623680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:32.625348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b354fa-58d0-4e51-afd6-957acca1fd02 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:32.674621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:32.676690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.677977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.678768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.693533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:32.703628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:32.704721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:32.705867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:32.708016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 79/156] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:32.713612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:32.750660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ae7b8-aa66-4fcf-8d58-c8ebe1fbf236 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:32.803060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:32.804010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.804308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.804562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.808706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:32.821090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:32.830769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:32.831729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:32.832488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:32.833826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 78/157] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:32.833916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:32.848018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aacc1-9935-4e44-bcbe-6912628467d0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:32.891093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:32.892001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:32.892290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:32.892573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:32.895078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:32.906817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:32.917538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:32.918608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:32.919482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 80/158] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:32.920111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:32.972548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096491-7c2b-43d6-84ab-aafd1d8e26bb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.021022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.022001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.022356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.022789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.023627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:33.023990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.025640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12eecf2-8b6b-48a7-b6f9-643fc144acff None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.035582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 79/159] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.037114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8537-3d78-403e-84ef-8aa1f76f4e5f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.037641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8537-3d78-403e-84ef-8aa1f76f4e5f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.038003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8537-3d78-403e-84ef-8aa1f76f4e5f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.039417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 80/160] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:33.039959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af8537-3d78-403e-84ef-8aa1f76f4e5f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.054486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9a3ca-7986-440d-9a31-d3d0f3e9810c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.094347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.095437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.095896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.096350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.097336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:33.097819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.099469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.108501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 81/161] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.109657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.128838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.175604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af8537-3d78-403e-84ef-8aa1f76f4e5f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.182795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.220179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.220931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.221156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.221426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.222246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:33.222556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.223942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad8d34-d9b3-42e1-bbe1-3429d65076eb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.233235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 81/162] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.234562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.234671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.234879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.238742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400bbc-201b-47a0-9803-0ff1da51932f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.273662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:33.275627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:33.277601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.279005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.279487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.279957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] PATH_INFO: `/v3/users/a90171f1014d4a6dbe696da11bc1dd7a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.282358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] RBAC: Authorizing `identity:get_user(user_id=a90171f1014d4a6dbe696da11bc1dd7a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.284173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.286447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 82/163] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/users/a90171f1014d4a6dbe696da11bc1dd7a => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.288142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.417770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5192d7-2083-467a-9cce-200e4011fece None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.421122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 82/164] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:33.421977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.475821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.485925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.514640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.515517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.515793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.516061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.519634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.521180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.523691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 83/165] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.524029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.530324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ebc60-dc74-4cd3-89b7-5a5abaaa8349 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.567977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.568888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.569170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.569440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] PATH_INFO: `/v3/users/d5275bfb53864285ae97a884e77aef1b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.570651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] RBAC: Authorizing `identity:get_user(user_id=d5275bfb53864285ae97a884e77aef1b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.572112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.574112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 83/166] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/users/d5275bfb53864285ae97a884e77aef1b => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:33.575102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.680202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82d75-af58-4b4e-a441-074e316b59bc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.700047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.718301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.719469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.719949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.720433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.731590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:33.740076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:33.741028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:33.742006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:33.744104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 84/167] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:33.745060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:33.750841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346d9e-6834-4a31-b6bc-ef6b87f5967f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.848992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c4f9c-6d47-4ea5-a12b-2c62980e348e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:33.852553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.853290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.853577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.853853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.885575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:33.886434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:33.886863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:33.887564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:33.936999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:33.958098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:33.959767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:33.994456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.024811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.026314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.135237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 85/168] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:34.135915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.159784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 84/169] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:34.165384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.268685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.274873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec98d-4b57-4824-a0fb-63b6f65720e4 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.308997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.313001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.313867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.314150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.314429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.316460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.319863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.319863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 86/170] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:34.324403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.324494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892319-7b38-41ea-a058-fe884dda4a87 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.417290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.417977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.418270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.418555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.419380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:34.419692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.421185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.442991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ce52-008b-4721-b387-0ab3fd24e50d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.446826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:34.447638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.477532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.478393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.478675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.478950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] PATH_INFO: `/v3/projects/2fa55a4eeb5545c09984059721a29075` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.480134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] RBAC: Authorizing `identity:get_project(project_id=2fa55a4eeb5545c09984059721a29075)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.481964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.483839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 87/172] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/projects/2fa55a4eeb5545c09984059721a29075 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:34.484568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.500934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.640187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4efd-304c-4d6b-b1fe-ddd8a22aecab None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.644616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733bff5-b02d-48d3-9e3a-65ce13cb64b6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.701154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.702031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.702543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.702819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.705159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.707181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.708932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 86/173] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:34.709693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.715841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.716771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.717053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.717327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/users/d5275bfb53864285ae97a884e77aef1b/roles/66db09fca51443c0ab982c5e5ce5941f` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.721085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, user_id=d5275bfb53864285ae97a884e77aef1b, role_id=66db09fca51443c0ab982c5e5ce5941f)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.723162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.750227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.795571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 88/174] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:34 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/users/d5275bfb53864285ae97a884e77aef1b/roles/66db09fca51443c0ab982c5e5ce5941f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:34.796423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.882257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a28095-ac8d-48eb-b33d-9efbf8bb9f30 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:34.943669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:34.944632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:34.944920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:34.945195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] PATH_INFO: `/v3/users/9d949c21d8c14c66b498639dd44ff225` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:34.947662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] RBAC: Authorizing `identity:get_user(user_id=9d949c21d8c14c66b498639dd44ff225)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:34.947871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:34.950296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:34.952851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 87/175] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/users/9d949c21d8c14c66b498639dd44ff225 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:34.953627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:34.996023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b177e-3a87-474e-90b5-61efbaa42cbb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.071728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.072642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.072930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.073212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.074063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=2fa55a4eeb5545c09984059721a29075, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:35.074377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:35.075847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:35.092487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 89/176] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=2fa55a4eeb5545c09984059721a29075&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:35.093741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:35.170507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:35.187758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d1131-dbc3-46bb-9e4d-7b632f3deb0a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.233358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.234051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.234403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.234749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.235687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=a90171f1014d4a6dbe696da11bc1dd7a) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:35.236067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:35.237749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a691e-6567-4143-af57-5e1973e300f4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:35.244710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 88/177] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:34 2022] GET /identity/v3/role_assignments?user.id=a90171f1014d4a6dbe696da11bc1dd7a&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:35.277797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfaf9-8710-4415-bcc1-e434114e3db3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.332291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.333013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.333299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.333574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.343926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:35.428968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:35.430456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:35.554757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2de5-8426-459b-b60e-ec8991662f3b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.554945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2de5-8426-459b-b60e-ec8991662f3b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.555213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2de5-8426-459b-b60e-ec8991662f3b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.556669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 89/178] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:35.563056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd13d-72c6-43e0-a335-17a7be394cfe None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.563356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd13d-72c6-43e0-a335-17a7be394cfe None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.563614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd13d-72c6-43e0-a335-17a7be394cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.591371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 90/179] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:35.592772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-aeadd13d-72c6-43e0-a335-17a7be394cfe None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:35.593992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-aeadd13d-72c6-43e0-a335-17a7be394cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:35.601722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:35.674542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:35.717257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 90/180] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:35.718614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95aafcd-7089-421e-aef0-b4d38d054d49 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.718904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95aafcd-7089-421e-aef0-b4d38d054d49 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.719180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95aafcd-7089-421e-aef0-b4d38d054d49 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.720797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 91/181] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:35.726633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.726923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.727203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.745755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:35.747260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:35.761722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11598a7-8313-493e-b9f8-9c228f220ddc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.818976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.819933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.820311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.820616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.822159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:35.824308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:35.827016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 91/182] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:35 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:35.829343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:35.842206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 92/183] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:35.843593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:35.864888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:35.947952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71dfe5-3933-4d7d-8ca5-cfef5e0b0cfe None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.956925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b6943-f74b-41b8-929e-958ef83baa7a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:35.958162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:35.987361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.988151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.988436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.988763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.989934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:35.991382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f28f7-21cd-445b-ac9e-dc502da2d9a6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:35.992736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:35.993211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 92/184] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:35.994373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7dc89-33fa-474a-a477-48b8edb4be5a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:35.994665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7dc89-33fa-474a-a477-48b8edb4be5a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:35.994946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7dc89-33fa-474a-a477-48b8edb4be5a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:35.996276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 93/185] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:35.996961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7dc89-33fa-474a-a477-48b8edb4be5a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.001441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.001441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.001617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.002515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=9d949c21d8c14c66b498639dd44ff225) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:36.003355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.006171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4685-2088-4729-b201-cc3f1d0862d5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.016599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 93/186] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:35 2022] GET /identity/v3/role_assignments?user.id=9d949c21d8c14c66b498639dd44ff225&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:36.016599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.016599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.016599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.040163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:36.041512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:36.119425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 94/187] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:36.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.136787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7dc89-33fa-474a-a477-48b8edb4be5a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.149817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.180889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.181709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.181992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.182268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.198790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:36.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:36.213515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:36.214552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:36.217304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 94/188] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:36.222567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.268238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.277636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2349c-7b63-4931-b661-3cbff124d823 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.327607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.328382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.329579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.329579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.329772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.331716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.361339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cad4b2-979b-4a83-8f13-43dc2169659b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.397147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.397891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.398178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.398456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.408646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:36.416964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:36.417969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:36.418949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:36.420886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 95/189] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:36 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:36.421720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.423280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 95/190] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:36.424102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.505294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.559569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.565990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674e7b8-d9d5-425c-8216-f47f963f8840 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.570542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbf327-c8f3-41a6-bc43-20bd69e91fcb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.608892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.609615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.610139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.610421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.611284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:36.611596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.614739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.625246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.626181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 96/191] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:36.627091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.627377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.627650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.629000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:36.629381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.630695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.631925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.640990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 96/192] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:36.646933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.768867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcba164-39b7-45a9-b87d-dc82a08b585a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.769092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8c9b3-581c-4ad2-8837-38ee85c8dd0b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.799364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.808889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.809405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.809772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.812750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.812858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.812933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.812999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.812999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:36.827351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.827636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:36.828845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.829063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:36.831585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 97/194] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:36.831724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 97/194] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:36.837726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.838739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:36.856832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:36.958309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf886e15-aa90-4251-a0b5-116796d644f5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.973288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6c39-d97d-4753-acfa-d51fc399dea0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:36.997800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:36.998507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:36.998722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:36.998992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.024799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:37.025802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.026286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.026758 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.095907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:37.131870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.133662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.157109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.158886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.180979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:37.273528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 98/195] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:37.278951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:37.380329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 98/196] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:37.382868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:37.386522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:37.405343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7efb7e-b301-43ab-99a3-a1b7aed05203 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:37.447645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:37.448943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.449676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.450128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.451624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.453424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.455787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 99/197] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:37 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:37.462289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:37.580834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efc0a-de6a-4979-bbe8-2f5cd69e58cb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:37.622686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:37.623519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459d4f1-1d6e-4942-81ee-e98e5e831e4c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:37.623805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.624232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.624683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.625950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=a034cf00fb3d4b929304c887a4f51792) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:37.626401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.628052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.636155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 100/198] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:37 2022] GET /identity/v3/role_assignments?user.id=a034cf00fb3d4b929304c887a4f51792&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:37.662694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:37.694111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:37.694984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.695265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.695538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.701587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.703079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d30d7-67f9-4dcf-aa5f-da0ad851a1a6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.705133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 99/199] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:37 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:37.711260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:37.713467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc3b2-aa71-432a-bc23-b8f180660021 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.713759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc3b2-aa71-432a-bc23-b8f180660021 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.714039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc3b2-aa71-432a-bc23-b8f180660021 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.715888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 100/200] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:37.721293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.721831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.722219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.745581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:37.760093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:37.762630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:37.850784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143e516-4cf7-4b3d-9767-3e3355e14085 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:37.893023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:37.895991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:37.895991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:37.895991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:37.895991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=a034cf00fb3d4b929304c887a4f51792) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:37.895991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:37.900963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:37.903536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 101/201] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:37 2022] GET /identity/v3/role_assignments?user.id=a034cf00fb3d4b929304c887a4f51792&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:37.912755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 101/202] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:37.935530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:38.040120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:38.069333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26885-a5ef-4cbf-b218-fd88e945f06c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:38.143104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:38.143895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:38.144180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:38.144458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:38.158726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:38.167679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:38.169025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:38.169842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:38.172500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 102/203] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:38.179202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:38.331559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ffe37-19e7-4692-9e07-759a3ba4ac07 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:38.345788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:38.378439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:38.379114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:38.379398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:38.379674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:38.380437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:38.380772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:38.382167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:38.396751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 102/204] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:38.402239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:38.507897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ace6a7-4ee1-433a-a69a-9e1c815e8dfd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:38.551145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:38.552027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:38.552313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:38.552604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] PATH_INFO: `/v3/users/d5275bfb53864285ae97a884e77aef1b` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:38.553847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] RBAC: Authorizing `identity:get_user(user_id=d5275bfb53864285ae97a884e77aef1b)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:38.555321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:38.557395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 103/205] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:38 2022] GET /identity/v3/users/d5275bfb53864285ae97a884e77aef1b => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:38.563025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:38.695201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771bc8-4d65-4b8e-af05-78d61c677c84 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:38.731429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:38.732372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:38.732824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:38.733240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:38.789354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:38.822970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:38.824453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:38.960051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 103/206] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:38.966040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:38.982620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:39.137435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0263de8-3af9-4c73-9f82-1ad6a1e9a9fb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:39.176392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:39.177290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.177575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.177848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.179044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:39.180495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaf2cb-e0ec-438f-ad9a-685c9ffc6788 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:39.182608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 104/207] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:38 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:39.188221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:39.301752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d1af-dec4-4af7-9a9f-5401489696a9 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.301944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d1af-dec4-4af7-9a9f-5401489696a9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.302218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d1af-dec4-4af7-9a9f-5401489696a9 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.303644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 105/208] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:39.322273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769909ef-09e5-48e2-8d8c-55133e4c7f88 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.322596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769909ef-09e5-48e2-8d8c-55133e4c7f88 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.322830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769909ef-09e5-48e2-8d8c-55133e4c7f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.347189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:39.371875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-769909ef-09e5-48e2-8d8c-55133e4c7f88 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:39.372829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42416e-5a5f-441c-91cd-b007646912a7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:39.373103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-769909ef-09e5-48e2-8d8c-55133e4c7f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:39.409051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:39.409719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.410013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.410354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.411251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, scope.project.id=529241aa434c450db6b72c5d75331f60, user.id=d5275bfb53864285ae97a884e77aef1b) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:39.411572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:39.413067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:39.426371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 104/209] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:39 2022] GET /identity/v3/role_assignments?user.id=d5275bfb53864285ae97a884e77aef1b&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:39.472995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 106/210] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:39.496410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:39.623749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e632e2-ab2a-4dea-bbd4-592ffd978700 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:39.673690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:39.674950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.675420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd336-6696-4157-b9e5-656efbb5cdb6 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.675641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.675762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd336-6696-4157-b9e5-656efbb5cdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.675971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd336-6696-4157-b9e5-656efbb5cdb6 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.676413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.677785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 107/211] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:39.686078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.686405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.686641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.689692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:39.698534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:39.699864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:39.700663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:39.703080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 105/212] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:39.703466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:39.704672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:39.715509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:39.716020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:39.716820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:39.805907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 108/213] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:39.839835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:39.873713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad888-4d77-4541-9a64-c7c95e8cfa9f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:39.930070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:39.930070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:39.930364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:39.930582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:39.931311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:39.931618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:39.935402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:39.945178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 106/214] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:39.950866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:39.967066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bffca-da81-4e36-99af-46d74dacdc75 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.021166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.022001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.022254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.022524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.024125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.028963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.028963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 109/215] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:40.034074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.047614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.049429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.099218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9edc2d-d54b-4330-8ddc-8f58d8201aa1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.136381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.137885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.144687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.144687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] PATH_INFO: `/v3/users/7cdeb65746934d7797e91461d0e6faa4` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.154548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] RBAC: Authorizing `identity:get_user(user_id=7cdeb65746934d7797e91461d0e6faa4)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.156645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.159691 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 107/216] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:39 2022] GET /identity/v3/users/7cdeb65746934d7797e91461d0e6faa4 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:40.163608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9abeee-9dc8-4973-bebe-ce0617b8e9d0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.166119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.202586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.203173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.203459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.203736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.204592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.206883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.281988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281f57b-b700-45d3-baa4-ed78aca8e86b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.302335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.322978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.323999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.324415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.324846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.329433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 110/217] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:40 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:07:40.406159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.407928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd36e8-554b-4583-a4d9-f58a1bedda05 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.519377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 108/218] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:40.525374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.590438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.593375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3efcdd-78b4-4fc4-b85b-fb074c6d261a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.593508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3efcdd-78b4-4fc4-b85b-fb074c6d261a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.594085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3efcdd-78b4-4fc4-b85b-fb074c6d261a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.595170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 109/219] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:40.601106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d798-4498-40fd-b870-9744c3b246e0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.601395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d798-4498-40fd-b870-9744c3b246e0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.601670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d798-4498-40fd-b870-9744c3b246e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.621000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4ff6d798-4498-40fd-b870-9744c3b246e0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:40.622259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4ff6d798-4498-40fd-b870-9744c3b246e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:40.637955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.644227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f92a61-4272-4eb7-a6dd-99dfcfd1c775 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.687785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.691127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.691127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.691127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.691378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.693255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33808895-14b2-455a-855b-b4b362947b33 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.695012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 111/220] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:40.698158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d222e-87b1-4bf4-9906-043f01b4782e None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.698446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d222e-87b1-4bf4-9906-043f01b4782e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.698716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d222e-87b1-4bf4-9906-043f01b4782e None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.700145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 112/221] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:40.701551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d222e-87b1-4bf4-9906-043f01b4782e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.701923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 110/222] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:40.706253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.707415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.707718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.727153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:40.728491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:40.821318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 111/223] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:40.828299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d222e-87b1-4bf4-9906-043f01b4782e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.828478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.868867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.880942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.881626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.882156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.882800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.883762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=7cdeb65746934d7797e91461d0e6faa4) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:40.884435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.886679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.895266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 113/224] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/role_assignments?user.id=7cdeb65746934d7797e91461d0e6faa4&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:40.896122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.928627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.962241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848143f-43f6-44da-90fd-06d5afa56176 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.034772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540376-d526-46a9-bc8a-470e9c11399b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.048464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.049615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.050059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.050478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.082343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:41.092168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:41.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:41.094370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 112/225] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:41.100010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.100940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.101920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.102369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.102784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.128254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:41.145426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:41.146391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:41.147161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:41.149264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 114/226] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:41.157633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.241801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d930a16-ca4d-4214-bbc1-c1ab495d0e31 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.272975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:41.290026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.290704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.290988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.291261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.292023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:41.292329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.293741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.308147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:41.317677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 113/227] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:41.323339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.350279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33115ec-401b-415f-9668-26a4b847942e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.385428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.386110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.386397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.386675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.387690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:41.388120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.389685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.399175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 115/228] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:41.405462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.512998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750ef0-8d5c-4885-96a9-26cd87187441 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.544969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:41.553273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.554115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.554402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.554682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.556073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.558121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.559790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 116/229] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:41.566259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.638870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458019cb-bb6f-4adf-8425-97fdf3d2ea17 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.681297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f08214-dddb-4b03-9918-cdd434c6b7be None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.718573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.719927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.720210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.720485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.722275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.723862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.725732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 117/230] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:41.731423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.748736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.749645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.749934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.750213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.751638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.753702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.755679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 114/231] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:41.765653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:41.794760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:41.874888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7ab95-23ab-4f44-8095-7fd8d4f4d284 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:41.906456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:41.930238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:41.931223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:41.931509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:41.931787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/a034cf00fb3d4b929304c887a4f51792/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:41.934646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, user_id=a034cf00fb3d4b929304c887a4f51792, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:41.936120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:41.957660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d4c85-1e89-42b4-b00d-9f200e10e49b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:42.000976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:42.001889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 118/232] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:41 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/a034cf00fb3d4b929304c887a4f51792/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:42.002002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:42.002218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:42.002494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:42.004803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:42.008268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe120cc5-67c9-45d2-878c-b64d2c646904 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:42.008268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 115/233] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:41 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:42.020861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:42.159703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ca071-63f0-470a-9d3f-35a89d45c71b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:42.175933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:42.211354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:42.212722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:42.213343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:42.213828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/a034cf00fb3d4b929304c887a4f51792/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:42.222390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, user_id=a034cf00fb3d4b929304c887a4f51792, role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:42.224361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1d34d-07f6-4017-99d7-b843f10aa0e2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:42.345448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 119/234] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:42 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/a034cf00fb3d4b929304c887a4f51792/roles/2c535dd04ec64d44949ee9193f6d696e => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:43.155744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43518f4-f4c7-4092-b86a-30ef0cb558af None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.156291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43518f4-f4c7-4092-b86a-30ef0cb558af None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.156291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43518f4-f4c7-4092-b86a-30ef0cb558af None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.158130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 116/235] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:43.163166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5205-2384-4ffa-af06-ed25e19771f4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.163683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5205-2384-4ffa-af06-ed25e19771f4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.164166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5205-2384-4ffa-af06-ed25e19771f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.193582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-48aa5205-2384-4ffa-af06-ed25e19771f4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:43.195105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-48aa5205-2384-4ffa-af06-ed25e19771f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:43.283586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 120/236] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:43.322325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43518f4-f4c7-4092-b86a-30ef0cb558af None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:43.398324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89722e-66c8-4ee3-8fab-39c6707063f8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.399290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89722e-66c8-4ee3-8fab-39c6707063f8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.399770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89722e-66c8-4ee3-8fab-39c6707063f8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.401419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 121/237] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:43.406518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.407029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.407508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.425454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:43.427000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:43.494401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43518f4-f4c7-4092-b86a-30ef0cb558af None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:43.500418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 122/238] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:43.526543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:43.560398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:43.561584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.563705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.564193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.569647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:43.574931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:43.581525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:43.589311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:43.590182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:43.590971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 117/239] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:43.602582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:43.656673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2306c-30e6-4e37-90a8-8ee1ae7b9c5c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:43.717398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:43.718553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.719040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.719924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.722199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:43.724794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760e043-6b93-4628-9ac6-9020d5a2059a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:43.727133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 123/240] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:43.733022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7ae24-550e-42bb-b418-8c9ad62b8e51 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.733022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7ae24-550e-42bb-b418-8c9ad62b8e51 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.733289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7ae24-550e-42bb-b418-8c9ad62b8e51 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.734692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 124/241] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:43.735252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7ae24-550e-42bb-b418-8c9ad62b8e51 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:43.741009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb166b-36db-4dd2-9fbd-dfe14fa0443e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:43.789083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:43.790587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.791094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.791841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.792887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:43.793664 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:43.795668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee38f65-5592-42d5-9602-da2bbb3ae749 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:43.807737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 118/242] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:43.812403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.812403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.812403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.835827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:43.837725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:43.851045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:43.856313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7ae24-550e-42bb-b418-8c9ad62b8e51 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:43.894234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:43.895115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:43.895491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:43.896226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:43.896842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:43.898511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:43.909855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:43.932535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 119/243] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:43.932535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.005817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 125/244] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:43 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:44.007446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.079594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae357d4-63be-487e-83e5-b538943e6b0f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.145061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398bfb-af9e-4d20-b738-1010f60432d5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.154328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.155225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.155510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.155789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] PATH_INFO: `/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.187581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] RBAC: Authorizing `identity:get_user(user_id=03fd30ddca4044b8bf5aab93b8d8b68c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.188269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.189108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.189550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.190566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.191134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.196606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 120/245] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:43 2022] GET /identity/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:44.198855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.209314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:44.219227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:44.220205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:44.221026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:44.223770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 126/246] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:44.229522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.269890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.270154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.355055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71293-b56f-47f5-a1f2-a639dfce1da3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.396528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72aac6-fa9b-48d6-95b9-da82ff244ea8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.398359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.399959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.400256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.400540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.442358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.443641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.443934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.444219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.445041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:44.445355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.447788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.458974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 127/247] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:44.465825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.491965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.493997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e65bc-a035-49fa-b6a2-75e42124c287 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.556122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.574511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.606654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94528c-df03-41a4-9755-1e1c71f34568 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.637681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 121/248] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:07:44.639969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb541-9277-461c-959a-4695b1f80f76 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.640573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb541-9277-461c-959a-4695b1f80f76 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.641023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb541-9277-461c-959a-4695b1f80f76 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.643838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 122/249] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:44.645017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb541-9277-461c-959a-4695b1f80f76 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.653301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.654193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.654482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.654767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] PATH_INFO: `/v3/users/7cdeb65746934d7797e91461d0e6faa4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.656093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] RBAC: Authorizing `identity:get_user(user_id=7cdeb65746934d7797e91461d0e6faa4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.657840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c031e1-cfe1-4df4-a410-ad0c768d5901 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.659813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 128/250] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/users/7cdeb65746934d7797e91461d0e6faa4 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:44.665279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.665624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.665843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.686100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:44.687311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:44.782918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 129/251] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:44.783811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.784864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb541-9277-461c-959a-4695b1f80f76 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.826887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.827887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.828115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.828399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.829859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.832008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.834252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 123/252] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:44.836021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:44.854626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.897020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:44.928748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df53715-ff7c-447f-9776-4bb1173e3e69 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.964034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb7f4-59f5-401a-8430-69a25aab72c0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:44.972255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:44.973428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:44.974362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:44.974815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:44.976209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:44.978401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:44.981430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 130/253] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:44.983161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.017329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.018392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.018500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.018788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.057541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:45.069350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:45.070266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:07:45.071059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:45.073165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 124/254] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:45.074155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.114802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06a425-8dd9-40d1-bb60-972d1dec1fb2 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.165172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.166099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.166523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.167063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.168043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=03fd30ddca4044b8bf5aab93b8d8b68c) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:45.168616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:45.170223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:45.185535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 131/255] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:44 2022] GET /identity/v3/role_assignments?user.id=03fd30ddca4044b8bf5aab93b8d8b68c&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:45.186763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.202018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:45.207476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:45.263198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c32085-a00a-47ec-a5c2-0611d35fa01a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.343703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.344662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.344949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.345230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/7cdeb65746934d7797e91461d0e6faa4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.350655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, user_id=7cdeb65746934d7797e91461d0e6faa4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:45.352263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd43a7e-2726-4a50-bfc2-3c60d50cbb09 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:45.357770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baccdf-fd1f-48db-8538-39614458b33a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.399914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.400971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.401462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.401887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.402793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:45.403294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:45.404860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e3c90-e024-48c3-a7db-e747d8bd0de9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:45.414776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 132/256] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:45.416565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1daec-f447-4b6e-8db0-c78b8515e3c3 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.417066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1daec-f447-4b6e-8db0-c78b8515e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.417492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1daec-f447-4b6e-8db0-c78b8515e3c3 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.418932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 133/257] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:45.420761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1daec-f447-4b6e-8db0-c78b8515e3c3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.433331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 125/258] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:45 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/7cdeb65746934d7797e91461d0e6faa4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:45.438296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.438296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.438296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.469206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:45.470638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:45.501209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:45.542478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1daec-f447-4b6e-8db0-c78b8515e3c3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.582163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.583609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.584127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.584639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.586217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:45.587038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 126/259] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:45.588296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:45.590563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 134/260] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:45 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:45.596686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.629824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.737238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:45.784335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568497-348a-483a-864c-9d1ee598c685 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.784606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc3468-49cd-4413-91ec-e9f1f68fc4f1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:45.822709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.823533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.823818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.824097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.826985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:45.837133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:45.845364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:45.846319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:45.847103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 135/261] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:45.852611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:45.878310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:45.878976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:45.879264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:45.879539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:45.975373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:45.976908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:45.997397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649c9-e743-41f7-b88c-7307c258b336 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:46.053255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:46.085284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:46.086230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:46.086656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:46.087074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:46.088077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:46.088528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:46.089192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 127/262] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:07:46.090569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:46.094803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:46.111096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 136/263] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:46.116880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:46.197930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b532-ff68-4581-b65d-cc92c0ce92b7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:46.234157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:46.235039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:46.235324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:46.235668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:46.237171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:46.238961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:46.241032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 128/264] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:46 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:46.247311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:46.270150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:46.310652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:46.366382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdad1b1-2a7e-4574-a25d-836946ed195c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:46.370046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a6b5-9fdf-45d5-b93e-3702ebdebb59 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:46.406564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:46.407276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:46.407569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:46.407849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:46.409248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=a034cf00fb3d4b929304c887a4f51792) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:46.409567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:46.411379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:46.418553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 129/265] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:46 2022] GET /identity/v3/role_assignments?user.id=a034cf00fb3d4b929304c887a4f51792&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:46.447192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:46.448112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:46.448415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:46.448700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] PATH_INFO: `/v3/users/a034cf00fb3d4b929304c887a4f51792` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:46.450013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] RBAC: Authorizing `identity:get_user(user_id=a034cf00fb3d4b929304c887a4f51792)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:46.451582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:46.453552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 137/266] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:46 2022] GET /identity/v3/users/a034cf00fb3d4b929304c887a4f51792 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:46.459299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:46.573950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:46.580160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bb296-515c-4520-9e67-48b7dd4882ff None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:46.618172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:46.618844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:46.619182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:46.619471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:46.707652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:46.709159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:46.825917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 130/267] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:46.826435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:46.833885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:46.983447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b0ec9-2180-4f37-a77d-d582d7aff375 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:47.016971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:47.026322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:47.027249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.027515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.027778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.029031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:47.030539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7abf71-6fd3-42f4-aadb-6a003684c150 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:47.032533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 138/268] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:46 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:47.042663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:47.179812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f89e9-548c-40f4-b0be-847834f9e619 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:47.228182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:47.229133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.229564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.229986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.231015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=a034cf00fb3d4b929304c887a4f51792) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:47.231476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:47.233117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba271c-73b8-42de-97d6-2a7333edafd0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:47.240029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 131/269] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:47 2022] GET /identity/v3/role_assignments?user.id=a034cf00fb3d4b929304c887a4f51792&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:47.607943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baefc5-b1c6-40ab-abdc-6c0d4477088b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.608143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baefc5-b1c6-40ab-abdc-6c0d4477088b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.608445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baefc5-b1c6-40ab-abdc-6c0d4477088b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.609944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 139/270] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:47.616210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.616804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.617306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.635044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:47.636557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:47.696798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:47.709469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 132/271] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:47.733668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baefc5-b1c6-40ab-abdc-6c0d4477088b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:47.841630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baefc5-b1c6-40ab-abdc-6c0d4477088b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:47.880514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:47.881292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.881577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.881854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.895147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-8373fcb2-e6d7-433a-b8eb-48c135493f03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:47.903297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:47.904252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:47.905127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 140/272] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:47.912450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:47.925124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbff8-a24b-4efd-bce7-017d03d54e7e None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.925395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbff8-a24b-4efd-bce7-017d03d54e7e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.925736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbff8-a24b-4efd-bce7-017d03d54e7e None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.927705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 141/273] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:47.933017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:47.933367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:47.933654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:47.956060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:47.957310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:48.036603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.044744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ca1f-3f99-4520-b524-0c0bdf588744 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.061271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 142/274] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:48.086039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.086930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.087612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.088011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.088295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.089136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:48.089450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.090871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.099802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 133/275] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:48.105799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.177462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.193224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0093d-26b0-4ec8-bfd2-3d55c505ae2a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.241279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.242125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.242405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.242674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.244540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.246908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf332-e801-4038-b073-9a547059bce9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.255009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 143/276] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:48.255009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7595-9ee1-4358-b137-cd5049f5e375 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.255009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7595-9ee1-4358-b137-cd5049f5e375 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.255283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7595-9ee1-4358-b137-cd5049f5e375 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.256387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 144/277] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:48.258068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3fc9a-372d-41ff-b714-2d93918191f7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.258526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b7595-9ee1-4358-b137-cd5049f5e375 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.291846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.292747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.293029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.293306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] PATH_INFO: `/v3/users/95db7af45dc04a71b97643d001afdf6a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.307373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] RBAC: Authorizing `identity:get_user(user_id=95db7af45dc04a71b97643d001afdf6a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.308842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7232b-27d8-47c3-8dbd-a564148b918d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.310705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 134/278] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/users/95db7af45dc04a71b97643d001afdf6a => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:48.311762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.312044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.312315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.323269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.330777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:48.332115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:48.374457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b7595-9ee1-4358-b137-cd5049f5e375 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.403436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 135/279] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:48.404411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.421202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.421977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.422785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.423152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.424085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.426316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.450636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.498322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 145/280] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:07:48 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:48.505915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.508151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ce0a-a304-4078-af82-365e26175a7d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.549959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.550937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.550937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.551128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.551970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, user.id=95db7af45dc04a71b97643d001afdf6a, scope.system=all) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:48.552304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.553753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a284-ce83-4c83-a2d5-21655c90283b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.559404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 136/281] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/role_assignments?user.id=95db7af45dc04a71b97643d001afdf6a&scope.system=all&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:48.560617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837eb0-1a8a-493a-b5b4-6cec447711e0 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.568647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837eb0-1a8a-493a-b5b4-6cec447711e0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.568647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837eb0-1a8a-493a-b5b4-6cec447711e0 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.568647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 137/282] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:48.571032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.571313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.571585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.590471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:48.591752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:48.668344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bd608-972b-43ba-b79a-a59ff8019c56 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.680436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 138/283] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:48.699194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.701018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.719354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.719354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.719622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.720001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.734754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:48.743255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:48.744213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:48.745192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 146/284] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:48.751455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.755778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:48.842238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb9fbe-eb93-4c5e-a74e-7ecc8c930f07 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.888924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b506cc-8657-4305-8279-afcc1e6699d3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:48.892309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.893744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.894029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.894303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.908122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:48.916072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:48.917043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:48.917828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:48.924645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 139/285] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:48.926705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.929567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:48.930252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:48.930537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:48.930816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:48.931585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:48.931892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:48.933858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:48.944087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 147/286] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:48.949794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:48.990444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:49.057174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:49.066599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c11d49-45c9-45c8-aad1-b8fcec08b1a8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.077328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712bfef-e7f5-4a7c-a1d3-fa0f0ba6c3ec None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.110750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.111716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.112149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.112568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.113491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:49.114046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.115596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.116023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.117051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.117434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.117808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] PATH_INFO: `/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.119237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] RBAC: Authorizing `identity:get_user(user_id=03fd30ddca4044b8bf5aab93b8d8b68c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.120834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.122898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 148/287] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:49.129804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:49.130066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 140/288] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:49.132440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:49.301765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:49.305559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5965-d8ab-4906-854e-b69a3c50546e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.343425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.344290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.344599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.344889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.346152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.346533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e793a-6ef7-4b6f-b03c-9790b8e332b5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.347646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.354585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 149/289] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:49 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:49.356317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:49.437858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.438738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.439018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.439284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] PATH_INFO: `/v3/users/7cdeb65746934d7797e91461d0e6faa4` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.445333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] RBAC: Authorizing `identity:get_user(user_id=7cdeb65746934d7797e91461d0e6faa4)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.446835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.448830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 141/290] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:49 2022] GET /identity/v3/users/7cdeb65746934d7797e91461d0e6faa4 => generated 297 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:49.454565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:49.475379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580a39-ff99-47ab-863f-6e0e5a75af60 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.510446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:49.541909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.542804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.543090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.543371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/03fd30ddca4044b8bf5aab93b8d8b68c/roles/66db09fca51443c0ab982c5e5ce5941f` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.546267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, user_id=03fd30ddca4044b8bf5aab93b8d8b68c, role_id=66db09fca51443c0ab982c5e5ce5941f)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.548206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.610923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20795786-39ab-4297-8264-96ad52266ffc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:49.641732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 150/291] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:07:49 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/users/03fd30ddca4044b8bf5aab93b8d8b68c/roles/66db09fca51443c0ab982c5e5ce5941f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:07:49.660005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:49.660730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:49.661017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:49.661293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:49.773632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:49.775133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:49.816711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:49.905948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 142/292] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:07:49.911671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:49.972918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:50.091389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628ebc-6bf0-4330-bf0a-5c56115b3da7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:50.141684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:50.142596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:50.142910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:50.143192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:50.149415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:50.150920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e9de-b4ee-414c-a8fc-191da2b98eec None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:50.153168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 151/293] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:49 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:50.160023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:50.262133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486811-62b5-40a1-849c-97df26e402d0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:50.293615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:50.310621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:50.311284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:50.311569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:50.311850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:50.312714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=7cdeb65746934d7797e91461d0e6faa4) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:50.313031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:50.314518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b74dd-1113-45bc-aae3-26428b81b2fc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:50.325109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 143/294] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:50 2022] GET /identity/v3/role_assignments?user.id=7cdeb65746934d7797e91461d0e6faa4&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:51.184137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5138c-74fd-4f5f-abe4-8b8f5e8212f3 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.184669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5138c-74fd-4f5f-abe4-8b8f5e8212f3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.184746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5138c-74fd-4f5f-abe4-8b8f5e8212f3 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.186100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 152/295] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:51.191501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.191900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.192064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.209873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:51.211058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:51.259892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735beb-6e27-419d-b33d-36b026d0b4de None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.260166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735beb-6e27-419d-b33d-36b026d0b4de None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.260383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735beb-6e27-419d-b33d-36b026d0b4de None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.263481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 153/296] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:51.267752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608297-780f-48c8-930c-849ad23ef3b8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.268563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608297-780f-48c8-930c-849ad23ef3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.268869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608297-780f-48c8-930c-849ad23ef3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.287066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 144/297] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:51.287369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bf608297-780f-48c8-930c-849ad23ef3b8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:51.288666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bf608297-780f-48c8-930c-849ad23ef3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:51.314838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.357663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 154/298] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:51.362063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0a58-ca2f-494c-af3f-8d1119b29f8c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.363113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0a58-ca2f-494c-af3f-8d1119b29f8c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.363597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0a58-ca2f-494c-af3f-8d1119b29f8c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.369799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 155/299] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:51.371214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.371499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.371772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.388728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:51.390055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:51.410425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.433334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae17c39-f25f-4c38-90da-b194115afeb9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.452468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.463371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 156/300] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:51.464065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.478796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.480286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.480819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.481377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.492533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:51.500333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:51.501235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:51.502027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:51.504884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 145/301] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:51.505814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.571058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e3753-c7d6-40a2-a555-0261dca19a54 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.607954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.609112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.609570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.609851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.618654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d79f56-06d0-4037-ab92-df9c2b7b1c9e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.619924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:51.628374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:51.632816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 157/302] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:51.633739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.648046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.666124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.666920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.667207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.667485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.668753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:51.670251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:51.671874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 146/303] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:51.672646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.722755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.747642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671810a-a59a-43fd-9341-84632e72c459 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.782134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff74f-d76d-4cf5-9e8c-345f26816faf None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.787510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.788384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.788866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.789317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.790205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:51.790681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:51.792212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:51.801371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 158/304] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:51.802301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.819279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.819956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.820242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.820517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.821364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:51.821658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:51.822977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:51.836200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 147/305] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:51.836200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:51.895222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.919470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e4fc8-1380-4a01-af98-f56d2d7d18a8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.934319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848a599-8e78-4f85-bc2a-7c0ac29d5189 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:51.943301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:51.956863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.957647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.957935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.959409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] PATH_INFO: `/v3/groups` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.959409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:51.960376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:51.969866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:51.970739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:51.971152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:51.971428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] PATH_INFO: `/v3/users/95db7af45dc04a71b97643d001afdf6a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:51.972661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] RBAC: Authorizing `identity:get_user(user_id=95db7af45dc04a71b97643d001afdf6a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:51.974092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:51.975715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 148/306] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/users/95db7af45dc04a71b97643d001afdf6a => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:51.977144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:52.043071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 159/307] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Jun 7 09:07:51 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:07:52.045307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:52.094645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a195fa-2aab-42fa-9384-a3aa636daffd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:52.135325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:52.136204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.136481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.137033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] PATH_INFO: `/v3/users/37cde6dd947244d4b330c0d160b0acf7` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.155263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] RBAC: Authorizing `identity:get_user(user_id=37cde6dd947244d4b330c0d160b0acf7)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:52.156790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:52.158830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 149/308] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:51 2022] GET /identity/v3/users/37cde6dd947244d4b330c0d160b0acf7 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:52.164618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:52.203081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:52.216658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5142f-3d0a-4b82-8ddd-5f27fc0ceb18 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:52.264662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:52.265806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.266239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.266671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] PATH_INFO: `/v3/system/users/95db7af45dc04a71b97643d001afdf6a/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.268763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=95db7af45dc04a71b97643d001afdf6a, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:52.270313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac0e7-b2db-49f2-b7f7-ebd2d0816dcf None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:52.275010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:52.278811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04496061-85ef-4bb4-851b-ddacb1b9780b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:52.318826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:52.319553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.319809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.320188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.321111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, user.id=37cde6dd947244d4b330c0d160b0acf7, scope.system=all) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:52.321422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:52.322845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0efe1a-189d-4ffe-a155-bce7b3f63438 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:52.328365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 150/309] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Jun 7 09:07:52 2022] GET /identity/v3/role_assignments?user.id=37cde6dd947244d4b330c0d160b0acf7&scope.system=all&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:52.367308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 160/310] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Jun 7 09:07:52 2022] PUT /identity/v3/system/users/95db7af45dc04a71b97643d001afdf6a/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:07:52.427910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2cd2c-6b6a-458e-974a-19c90e12ac68 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.428241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2cd2c-6b6a-458e-974a-19c90e12ac68 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.428519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2cd2c-6b6a-458e-974a-19c90e12ac68 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.430088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 151/311] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:52.436207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.436517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.436849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.457535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:52.458774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:52.543649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 161/312] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:52.567683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2cd2c-6b6a-458e-974a-19c90e12ac68 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:52.683035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:52.687898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2cd2c-6b6a-458e-974a-19c90e12ac68 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:52.724116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:52.724936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.725215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.725489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.736227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:52.744613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:52.745494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:52.746518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 152/313] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:52.751537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:52.859162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aefd6-41e6-450d-9124-71202ab09781 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:52.891118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:52.898307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:52.902956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:52.912511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 162/314] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:52.917735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:53.026008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e70a-6363-461b-837f-e706870f1ed2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:53.061620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:53.062486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:53.062766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:53.063044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] PATH_INFO: `/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:53.064239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] RBAC: Authorizing `identity:get_user(user_id=03fd30ddca4044b8bf5aab93b8d8b68c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:53.066240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:53.068165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 153/315] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:52 2022] GET /identity/v3/users/03fd30ddca4044b8bf5aab93b8d8b68c => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:53.073635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:53.181556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4bce-2b5a-46ad-a25e-49ba12fca97e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:53.216678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:53.218125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:53.218586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:53.219037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:53.283208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:53.293417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:53.294908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:53.397655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 163/316] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:53.405858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:53.435563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:53.513419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c19d7-3e29-46c0-b336-b360864d223f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:53.550136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:53.551348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:53.551828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:53.552295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:53.553741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:53.555491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a558bb0-a516-4283-b2bb-6b6b2c12ab1f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:53.557725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 154/317] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:53 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:53.563596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:53.661335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:53.671053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233821-782f-4f42-899a-e18b6373ef71 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:53.706481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:53.707140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:53.707422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:53.707693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:53.708525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91, user.id=03fd30ddca4044b8bf5aab93b8d8b68c) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:53.708864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:53.710304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db1d69-a677-4dcb-b3f9-d8e7132c02bb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:53.716600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 164/318] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:07:53 2022] GET /identity/v3/role_assignments?user.id=03fd30ddca4044b8bf5aab93b8d8b68c&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:54.376960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a1a6a-332b-4e43-bf00-2bdc2ab3d4f4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.377259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a1a6a-332b-4e43-bf00-2bdc2ab3d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.377545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a1a6a-332b-4e43-bf00-2bdc2ab3d4f4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.378891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 155/319] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:54.384007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c19f2-ee40-4224-b491-0f11f0d30863 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.384468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c19f2-ee40-4224-b491-0f11f0d30863 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.384547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c19f2-ee40-4224-b491-0f11f0d30863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.401751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-172c19f2-ee40-4224-b491-0f11f0d30863 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:54.402932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-172c19f2-ee40-4224-b491-0f11f0d30863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:54.470604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 165/320] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:54.493644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a1a6a-332b-4e43-bf00-2bdc2ab3d4f4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:54.589379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:54.597863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a1a6a-332b-4e43-bf00-2bdc2ab3d4f4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:54.618232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6ce2-480a-4748-b2ff-f242ed836878 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.618545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6ce2-480a-4748-b2ff-f242ed836878 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.618798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6ce2-480a-4748-b2ff-f242ed836878 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.620177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 166/321] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:07:54.625061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.625353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.625639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.633080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:54.633799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.634074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.634367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.635514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:54.637240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:54.638845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 156/322] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:07:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:54.643646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:54.644920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:54.645126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:54.716296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 167/323] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:54.739033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:54.750638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b629-bf0a-4b47-984d-548fcf0fa7ff None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:54.757102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:54.786920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:54.787624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.787908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.788173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] PATH_INFO: `/v3/groups` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.788914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:54.790332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76300acf-11f8-431c-a547-326dfb068c85 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:54.841342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68959-8f51-4cd1-af78-09db5c5fa956 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:54.863624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 157/324] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Jun 7 09:07:54 2022] POST /identity/v3/groups => generated 226 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:07:54.865813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a305-4400-46e2-bd69-54e21ea36e71 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.865813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a305-4400-46e2-bd69-54e21ea36e71 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.865813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a305-4400-46e2-bd69-54e21ea36e71 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.866757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 158/325] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:54.872952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.873397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.873739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.884922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:54.885661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:54.885944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:54.886220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:54.892989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:54.894416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:54.899048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:54.907013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:54.908046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:54.908900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 168/326] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:54.914452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:54.916718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:54.975812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 159/327] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:54.999330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.004734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:55.039495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35660-febb-42cb-b92f-54897c7725ee None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.078168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.078831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.079111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.081519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.081691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:55.082005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.083518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.097595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 169/328] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:55.098957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.123927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d0c88-6ee2-417f-867d-0a57be6e6a92 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.172884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.173687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.174489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.174758 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.187168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:55.194943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:55.195848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:55.196842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 160/329] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:55.202675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.221788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:55.226406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad283f66-4aff-45c9-93c6-d7d207305b6a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.268623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.269404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.269735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.272620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] PATH_INFO: `/v3/users/37cde6dd947244d4b330c0d160b0acf7` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.272620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] RBAC: Authorizing `identity:get_user(user_id=37cde6dd947244d4b330c0d160b0acf7)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.274070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.276673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 170/330] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:55 2022] GET /identity/v3/users/37cde6dd947244d4b330c0d160b0acf7 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:55.281921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.292893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:55.320951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b453306-64ee-46db-8d99-f4eaecb9c8c6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.360693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.361418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.362393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.362393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.362760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:55.363064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.364469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.374617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 161/331] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:55.382451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.411397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbe4f7-5f9f-40b6-92f4-1949367ff373 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.449510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.450286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.450566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.450875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] PATH_INFO: `/v3/system/users/37cde6dd947244d4b330c0d160b0acf7/roles/66db09fca51443c0ab982c5e5ce5941f` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.452566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=37cde6dd947244d4b330c0d160b0acf7, role_id=66db09fca51443c0ab982c5e5ce5941f)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.453972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.486763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:55.513855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9643-c6c5-4f71-81ec-9202068a4bf5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.515625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 171/332] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Jun 7 09:07:55 2022] PUT /identity/v3/system/users/37cde6dd947244d4b330c0d160b0acf7/roles/66db09fca51443c0ab982c5e5ce5941f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:07:55.566231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.567596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.568087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.568624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] PATH_INFO: `/v3/users/95db7af45dc04a71b97643d001afdf6a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.570326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] RBAC: Authorizing `identity:get_user(user_id=95db7af45dc04a71b97643d001afdf6a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.572167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94442ccf-c517-477b-a487-0643ba0bc8cd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.574605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 162/333] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:55 2022] GET /identity/v3/users/95db7af45dc04a71b97643d001afdf6a => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:07:55.580761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:55.666932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:55.704486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61fa68-b326-42cc-8798-cf287916569b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:55.751365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:55.752662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:55.752959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:55.753228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:55.754189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] RBAC: Adding query filter params (role.id=da277faf5b1141e1aaa9bdaf1713ae18, user.id=95db7af45dc04a71b97643d001afdf6a, scope.system=all) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:55.754511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:55.755981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d1e4-6a4b-4c56-9bff-0a9859b60103 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:55.769509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 172/334] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Jun 7 09:07:55 2022] GET /identity/v3/role_assignments?user.id=95db7af45dc04a71b97643d001afdf6a&scope.system=all&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:57.442121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f411-ce69-42ac-8689-17aa81bd96df None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:57.443153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f411-ce69-42ac-8689-17aa81bd96df None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:57.443643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f411-ce69-42ac-8689-17aa81bd96df None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:57.445517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 163/335] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:57.451369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:57.452242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:57.452773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:57.476878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:57.478419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:57.566133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 173/336] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:57.609517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037f411-ce69-42ac-8689-17aa81bd96df None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:57.735171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037f411-ce69-42ac-8689-17aa81bd96df None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:57.771949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:57.773155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:57.773645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:57.774145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:57.786101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:57.794211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:57.795112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:57.795893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:07:57.798724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 164/337] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:57.805319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:57.869688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:57.914054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fac90b-3428-4de2-be20-f2ce6c979cee None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:57.960689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:57.961794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:57.962290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:57.962832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:57.963845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:57.964405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:57.966138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:57.981896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 174/338] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:57.988562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:57.992727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.109710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4969e58-7bca-4d05-81c1-706f5bf85c1c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.147211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.147906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.148189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.148467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.220142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.222245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21f41d-e2b6-40ce-a265-52440c0690f2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.224266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.327091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 165/339] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:58.332781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.334684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f582-935c-41c7-826e-d43f6c96415a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.336437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f582-935c-41c7-826e-d43f6c96415a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.336437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f582-935c-41c7-826e-d43f6c96415a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.338596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 166/340] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:58.344897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.346382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.346857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.372634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:58.374417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:58.458667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8fab6-d02d-4e9c-bc9d-3edacf3b982d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.471257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 167/341] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:07:58.486615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.505116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.506032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.506311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.506582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.508018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.509549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.511813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 175/342] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:58.517875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.544520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.559504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.645212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19ffa4-abc7-4d16-b7df-29ab5b9f6b73 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.674471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b135c1-2aec-4f02-b231-fb537e57b649 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.684071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.685165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.685609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.686027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.695198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.697358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bc2af-c3f1-49e5-b2a5-59d4a41df19a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.699982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 168/343] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:58.701409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645af3f1-d0e1-4ec4-bfca-12369d356530 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.701828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645af3f1-d0e1-4ec4-bfca-12369d356530 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.702249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645af3f1-d0e1-4ec4-bfca-12369d356530 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.703633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 169/344] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:58.704849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a92dc-389c-45a0-9a61-cc596e316bc0 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.706227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a92dc-389c-45a0-9a61-cc596e316bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.706810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a92dc-389c-45a0-9a61-cc596e316bc0 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.708497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 170/345] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:07:58.709918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a92dc-389c-45a0-9a61-cc596e316bc0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.720963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.722127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.722588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.723039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.733983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c69c923d-3255-4ecd-82f8-a5962de4eff4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:58.743149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:58.744032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:07:58.744915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:58.746261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 176/346] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:07:58.747546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95949-aa1e-40a6-8a62-2a0df3ab2859 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.753166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95949-aa1e-40a6-8a62-2a0df3ab2859 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.753381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95949-aa1e-40a6-8a62-2a0df3ab2859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.771245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f0e95949-aa1e-40a6-8a62-2a0df3ab2859 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:58.772575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f0e95949-aa1e-40a6-8a62-2a0df3ab2859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:58.831610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.838970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.849307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a92dc-389c-45a0-9a61-cc596e316bc0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.861716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 177/347] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:58.863443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.863733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.864012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.887972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:07:58.889181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:58.896260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.897388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.897879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.898423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.899513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=529241aa434c450db6b72c5d75331f60) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:58.900102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.902342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.910063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 171/348] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:58.911261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.024460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 178/349] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:07:59.025373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.144372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6ee00-0042-4445-aeb0-2bb89cc5b565 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.156479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:59.190292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.191104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.191382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.191655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.194372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b7b7-15f2-449e-933e-588b113b0369 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.218698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:59.226620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:59.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:59.228848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 179/350] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:59.230591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.268479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.269187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.269471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.269747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.270505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:59.270817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:59.272190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:59.294973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 172/351] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:59.295662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.350531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:59.385012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8fb1b-6474-4f0e-a97b-d70c7208b2e9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.446597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.447674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.448119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.448542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.471766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:07:59.480217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:07:59.481329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 09:07:59.482295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:07:59.491885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 180/352] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:07:59.497791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.499900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04267739-737d-4a3d-a638-e926db58f82a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.537441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:59.551481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.552649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.552982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.553403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.554317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:59.554772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:59.556312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:59.566728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 173/353] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:59.567758 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.644747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11e4ff-410d-4293-9871-7c08b6e6248d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.686999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.687655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.687937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.688209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.705891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:59.722685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085b2c-6fd9-4b6e-a191-034b3138c565 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.760060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.761127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.761827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.762262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.763158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:07:59.763720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:59.765259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:59.775471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 174/354] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:07:59.776491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.789249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:59.790919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:59.826169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:59.923106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238b077-beb5-48a8-81e4-c9fc6921d743 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.970466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 181/355] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:07:59.973920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.974870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.976093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.976670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.977166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.027905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.093461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.095309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4886-5269-4b00-b46d-308bee21e9d6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.130682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b898c-a62f-4631-9d89-893df5324f19 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.157643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.165639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.166544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.166830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.167100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] PATH_INFO: `/v3/users/37cde6dd947244d4b330c0d160b0acf7` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.168489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] RBAC: Authorizing `identity:get_user(user_id=37cde6dd947244d4b330c0d160b0acf7)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.170100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.171892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 182/356] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/users/37cde6dd947244d4b330c0d160b0acf7 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.178327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.217441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 175/357] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:00.219244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0163b8-6ba5-4e92-b5b4-63b5e916e7da None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.219754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0163b8-6ba5-4e92-b5b4-63b5e916e7da None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.220230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0163b8-6ba5-4e92-b5b4-63b5e916e7da None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.222205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 176/358] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:00.223169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0163b8-6ba5-4e92-b5b4-63b5e916e7da None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.289060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.306079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c22ccf-a8ae-4b1e-b4da-9855aa178690 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.346419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0163b8-6ba5-4e92-b5b4-63b5e916e7da None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.348462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.349568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.349999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.350419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.351946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.353583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5adc8-8362-46b1-b0c2-ec3261bea3ee None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.355788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 183/359] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.358100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.358621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.359100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.377465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:00.379012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:00.388966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.389823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.389935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.390200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.391036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] RBAC: Adding query filter params (role.id=66db09fca51443c0ab982c5e5ce5941f, user.id=37cde6dd947244d4b330c0d160b0acf7, scope.system=all) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:00.391350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.393434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.406092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 177/360] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/role_assignments?user.id=37cde6dd947244d4b330c0d160b0acf7&scope.system=all&role.id=66db09fca51443c0ab982c5e5ce5941f => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.406928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.446966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.457863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 184/361] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:00.459203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.543376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2779f-4477-42b8-b0d4-50f8b73a197e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.547581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.588762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.589948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.590417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.590853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.592570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.594353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efaf8c-a473-4a72-a14b-b29d6e40079a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.603022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 178/362] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.604975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee308-92a1-46f6-84d9-bba916cb15f9 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.605505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee308-92a1-46f6-84d9-bba916cb15f9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.605974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee308-92a1-46f6-84d9-bba916cb15f9 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.607568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 179/363] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:00.608594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ee308-92a1-46f6-84d9-bba916cb15f9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.616803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5cdff-d4d1-408c-bf05-300220bba3b2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.664151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.665495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.665859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.666212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.668283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.669843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.672077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 185/364] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.673333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.749957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.753370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ee308-92a1-46f6-84d9-bba916cb15f9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.799388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42a5f-a29b-48a5-af5e-31012a12a9af None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:00.800622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.802575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.802864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.803141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.821194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-6aee369d-a553-4897-8319-3c686ac6bdfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:00.832695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:00.833618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:00.834445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:00.835815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 180/365] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:00.837278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.837575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.837853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.851244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:00.852110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:00.852390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:00.852747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:00.853975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:00.856285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:00.858898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 186/366] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:00.859695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:00.868864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:00.870603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:00.903991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:00.957116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 181/367] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:00.957812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.006706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb814-50e7-46ef-8c9f-b2bf69c87e20 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.038537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.066765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.066765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.066765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.066765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.067359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=529241aa434c450db6b72c5d75331f60) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:01.067359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.067359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:01.067764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 187/368] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:01.068721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.092212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de24b7-f110-4813-bf59-586e30c9c1b4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.140034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.140743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.141030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.141306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.142066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:01.142752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.143813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:01.160724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 182/369] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:01.162734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.204202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.217175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f86ae-ccca-4287-af5d-795dcaf852e2 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.260314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.261356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.262097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.262553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.263626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:01.264150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.266803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:01.274425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 188/370] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:01.275781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.344805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632beb86-0360-4c8f-a5e3-c534d50f0427 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.391704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.393432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.393868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.394320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.395415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.408189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:01.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:01.417647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:01.418435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:01.421283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 183/371] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:01.428161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.510949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3ee5-ca73-4b8b-bc86-de997e57da4b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.534461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.545773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984701-1728-4a0b-8911-2edaf3111184 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.593757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.595280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.596130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.597098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.602258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.603147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.603638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.604089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.605056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:01.606528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.609274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:01.629593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 184/372] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:01.654377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.736455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.738572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e58da1-64e7-4625-ba85-6c646610044b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:01.810428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.830610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e18c8-140a-4827-b97a-b5e0449aed5d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:01.876330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:01.876996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.877281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.877558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.899895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 189/373] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:01.901283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687a94-1759-4e0d-bb28-534ed0462915 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:01.901651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687a94-1759-4e0d-bb28-534ed0462915 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:01.902044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687a94-1759-4e0d-bb28-534ed0462915 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:01.903570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 190/374] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:01.907459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:01.909074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687a94-1759-4e0d-bb28-534ed0462915 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:01.977347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:01.979075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.051758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687a94-1759-4e0d-bb28-534ed0462915 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.093936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.094798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.095075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.095345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.097100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.098549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708f54f-9b69-46bd-83c2-6b4f4eadccae None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.100258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 191/375] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.101469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.101748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.102016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.120188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 185/376] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:02.121387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.121760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:02.122942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:02.145129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.174884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.197509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 192/377] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:02.198574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.235046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326d50-4413-4ae9-abc9-853b59ed5d23 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.272401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.273674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.274177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.274658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.276037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.277734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.279582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 186/378] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.280561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.304959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff7070-c28a-448b-97b7-fcd3b1cc7519 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.342522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.343190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.343476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.343748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] PATH_INFO: `/v3/projects/047614873c7a462ea30132f8c49e5842` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.344955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] RBAC: Authorizing `identity:get_project(project_id=047614873c7a462ea30132f8c49e5842)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.346488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.348573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 193/379] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/projects/047614873c7a462ea30132f8c49e5842 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.349352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.377207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.392004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ba0b7-6a57-4ab8-b7c0-d442ee8e16a9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.407623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.432845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.434068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.434566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.435051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.447689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:02.456992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:02.457858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:02.458656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:02.459257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cb264-489e-4f97-a738-c7e6ed81ba2b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.461564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 187/380] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:02.462615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.494903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.495558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.495839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.496113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.496968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:02.497276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.498812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.505030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 194/381] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.505843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.582000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccf35b-0ecf-4755-8c18-b1dd799b8356 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.604054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.620690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd9e8-6761-49f2-b7c4-123c40da4641 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.623756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.625002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.625500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.625989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] PATH_INFO: `/v3/groups/664cc8922772431d980c7664fa956650` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.632130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] RBAC: Authorizing `identity:get_group(group_id=664cc8922772431d980c7664fa956650)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.633978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.635936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 188/382] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/groups/664cc8922772431d980c7664fa956650 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.643009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.671411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.672366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.672826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.673268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.674775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:02.675272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.676969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.687908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 195/383] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:02.700827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.729862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:02.816388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e860-f6ee-42b9-b135-0c175b916825 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.840502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d752430-8ecf-469e-9fee-3f40a23602cd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:02.863584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.864929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.866855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.867958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.870978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.872661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.874771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 196/384] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:02.882751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:02.912694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:02.913501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:02.914250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:02.914250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:02.915211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] RBAC: Adding query filter params (group.id=664cc8922772431d980c7664fa956650, role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=047614873c7a462ea30132f8c49e5842) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:02.915626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:02.917391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:02.927373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 189/385] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/role_assignments?group.id=664cc8922772431d980c7664fa956650&scope.project.id=047614873c7a462ea30132f8c49e5842&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:02.930703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:03.008027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d616-7a56-4f85-8691-2e1dba8095ef None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:03.044081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:03.045002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:03.045289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:03.045567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:03.046906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:03.048416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd07ad-dfa4-4d7c-9224-10f10d6ee3af None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:03.051214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 197/386] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:02 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:03.056594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:03.170970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cff09-6a2a-4ef0-a17d-b81ce205e45f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:03.172862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:03.214555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:03.215368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:03.215649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:03.215923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/groups/109cfde66d214ed097ebf6afaa269f96/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:03.218238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, group_id=109cfde66d214ed097ebf6afaa269f96, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:03.220087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544d760-851f-430c-87cb-1d65e9ddc748 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:03.309844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 190/387] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Jun 7 09:08:03 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/groups/109cfde66d214ed097ebf6afaa269f96/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:03.890230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3def0-ab6f-4784-be79-b70ec6af2998 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:03.890760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3def0-ab6f-4784-be79-b70ec6af2998 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:03.890845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3def0-ab6f-4784-be79-b70ec6af2998 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:03.892887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 198/388] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:03.898488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed938fd-2dad-4201-bd8a-61856b8b0049 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:03.899369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed938fd-2dad-4201-bd8a-61856b8b0049 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:03.899858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed938fd-2dad-4201-bd8a-61856b8b0049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:03.925566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8ed938fd-2dad-4201-bd8a-61856b8b0049 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:03.927067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8ed938fd-2dad-4201-bd8a-61856b8b0049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:04.013586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 191/389] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:04.040070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d852dc-1503-4037-9d7d-948a582e234f None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.040365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d852dc-1503-4037-9d7d-948a582e234f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.040655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d852dc-1503-4037-9d7d-948a582e234f None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.043620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 199/390] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:04.048050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.054060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.054060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.054217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d852dc-1503-4037-9d7d-948a582e234f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.077625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:04.078859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:04.189115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:04.216019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 192/391] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:04.221837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d852dc-1503-4037-9d7d-948a582e234f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.264177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.264998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.265283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.265555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.282224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.300041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:04.307754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:04.308708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:04.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:04.321587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 200/392] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:04.324972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:04.330049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.471843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bedd7-94b1-475c-8309-841e6e499a9d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.478734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6edfb-9f57-4259-a8ad-90ed77c3a560 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.516462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.517232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.517557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.517842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.518973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:04.519300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:04.521116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:04.532632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 201/393] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:04.543335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.557393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:04.569151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.579516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.579516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.579516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.592860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:04.602792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:04.603704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:04.604474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 193/394] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:04.610020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.704815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:04.723885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d74396-da34-4ccf-adf2-1f0292fd29ee None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.760232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5275f91-76ac-4845-a022-9302bbf2b2b8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.761109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.762145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.762436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.762733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.763469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:04.763779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:04.765197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:04.774228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 194/395] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:04.780077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.877830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.878709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.879018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.879292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.880508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:04.882047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:04.883983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 202/396] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:04.889746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.892870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a066c-db2d-4f12-af8c-f633d04c7a74 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:04.932449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30179201-5832-4484-9e4d-214486807cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:04.933503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30179201-5832-4484-9e4d-214486807cdc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:04.933796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30179201-5832-4484-9e4d-214486807cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:04.934109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30179201-5832-4484-9e4d-214486807cdc None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:04.935388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30179201-5832-4484-9e4d-214486807cdc None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:04.936935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30179201-5832-4484-9e4d-214486807cdc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:04.939031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 195/397] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:04.944467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30179201-5832-4484-9e4d-214486807cdc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:04.966513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:05.056412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30179201-5832-4484-9e4d-214486807cdc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:05.084623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:05.093089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:05.093939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.094226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.094506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.097994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:05.097994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:05.101890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 196/398] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:05.105410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:05.125186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736393-1bbc-4e74-8c0f-22430e463347 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:05.194235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:05.195458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.195960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.196426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:05.196932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.203938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:05.205751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:05.208131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 203/399] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:04 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:05.213821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:05.217156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce2bb1-09bd-48a9-a1aa-b3ab34dea3bd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:05.255448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:05.256506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.256816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.257203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60/groups/109cfde66d214ed097ebf6afaa269f96/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.259594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=529241aa434c450db6b72c5d75331f60, group_id=109cfde66d214ed097ebf6afaa269f96, role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:05.268608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324c2de-4932-4d74-a3dd-4ddc1704aaf7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:05.337686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 197/400] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Jun 7 09:08:05 2022] PUT /identity/v3/projects/529241aa434c450db6b72c5d75331f60/groups/109cfde66d214ed097ebf6afaa269f96/roles/2c535dd04ec64d44949ee9193f6d696e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:08:05.358359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76f76-08ff-4340-8ebe-40ba5844a9b3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:05.408001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:05.408984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.409363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.409648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/groups/109cfde66d214ed097ebf6afaa269f96/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.411626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, group_id=109cfde66d214ed097ebf6afaa269f96, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:05.413185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826decf-9399-47ca-b2cd-c8e78a0674e9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:05.488141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 204/401] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Jun 7 09:08:05 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/groups/109cfde66d214ed097ebf6afaa269f96/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:08:05.792898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0788e-c680-4400-8dfe-8d0312902017 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.793272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0788e-c680-4400-8dfe-8d0312902017 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.793468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0788e-c680-4400-8dfe-8d0312902017 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.795151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 198/402] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:05.800507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.800830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.801103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.807892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:05.821269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:05.822458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:05.895264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead3bd4-d1a1-423c-b893-b4e1e6a2ac6d None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.895955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead3bd4-d1a1-423c-b893-b4e1e6a2ac6d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.896500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead3bd4-d1a1-423c-b893-b4e1e6a2ac6d None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.898103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 199/403] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:05.904234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:05.904528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:05.904842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:05.913101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 205/404] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:05.935786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:05.937595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:05.938799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:06.052208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 200/405] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:06.078360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.170117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2009e8-531b-4a8f-a3e9-bfc770872fef None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.190698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.235758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.237457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.237961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.238442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.240894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.244306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0104a-7ec8-425e-a22b-db2c4a86eac2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.261736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:06.276554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:06.277512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:06.278298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:06.282447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 206/406] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:06.288896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.318094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.318832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.319118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.319397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.339403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:06.351096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:06.351978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:06.352775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:06.356016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 201/407] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:06.363626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.430818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74985b-e6e8-4dea-9a5e-d8ac2c426a69 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.466696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.467715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.468282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.468894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.469836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:06.470904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:06.472374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4fb24-c787-4847-afb7-1ae80bfec774 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:06.483931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 207/408] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:06.483931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161327e-48a2-45db-9925-0cb1ebd55a7f None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.483931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161327e-48a2-45db-9925-0cb1ebd55a7f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.483931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161327e-48a2-45db-9925-0cb1ebd55a7f None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.483931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 208/409] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:06.488343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161327e-48a2-45db-9925-0cb1ebd55a7f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.516604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87579bc5-19f6-4189-b910-cd15acd159fb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.548947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.556460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.557183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.557467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.557743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.558501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:06.558808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:06.560804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1720c-61bc-4c46-9ee8-d8e6590ebf0c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:06.563920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.582874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 202/410] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:06.584493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.584968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.585390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.607602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:06.609828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:06.619829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161327e-48a2-45db-9925-0cb1ebd55a7f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.662437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.663902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.664285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.664715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.666629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:06.668441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:06.675587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 209/411] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:06.675587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.691001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 203/412] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:06.696821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.793793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7268e-1e91-411d-8144-499e9004ba99 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.819544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.821606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acee1e6-02fd-4a57-8487-7b6db4bef0db None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:06.832563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.839018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.839018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.839018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.839018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:06.839018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:06.839454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 204/413] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:06.844381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.856944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:06.871663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:06.873072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:06.873961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:06.874621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] PATH_INFO: `/v3/groups/664cc8922772431d980c7664fa956650` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:06.876436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] RBAC: Authorizing `identity:get_group(group_id=664cc8922772431d980c7664fa956650)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:06.878911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:06.886163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 210/414] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/groups/664cc8922772431d980c7664fa956650 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:06.886163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:06.971363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e788f82-74be-44b5-8f30-df2c5d498104 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.005160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48a02-df7d-4315-8f7a-2d592d91f372 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.021979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.022778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.023061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.023332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.036128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:07.044987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:07.046065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:07.046842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:07.049129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 205/415] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:07.050161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:07.056268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.058153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.058438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.058714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/groups/109cfde66d214ed097ebf6afaa269f96/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.061019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4e544e9f0c4114b0854e1efce46c91, group_id=109cfde66d214ed097ebf6afaa269f96, role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.062574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.106951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:07.133088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 211/416] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Jun 7 09:08:06 2022] PUT /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91/groups/109cfde66d214ed097ebf6afaa269f96/roles/2c535dd04ec64d44949ee9193f6d696e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:07.136723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:07.188396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd7686-99ab-4fec-8526-36e6e229ec62 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.249036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:07.255256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.256917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.257218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.257621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] PATH_INFO: `/v3/projects/047614873c7a462ea30132f8c49e5842` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.261096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] RBAC: Authorizing `identity:get_project(project_id=047614873c7a462ea30132f8c49e5842)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.262854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.265861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 206/417] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:07 2022] GET /identity/v3/projects/047614873c7a462ea30132f8c49e5842 => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:07.274081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:07.342791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeef70-0682-4984-882a-201649af78e6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.386045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.387050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.387333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.387602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.389067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:07.389368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.390796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.400771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 212/418] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:08:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:07.411190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:07.470176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8369-be1a-4a7b-8e70-527d7f57f5bd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.495184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:07.518949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.520101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.520386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.520687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] PATH_INFO: `/v3/projects/047614873c7a462ea30132f8c49e5842/groups/664cc8922772431d980c7664fa956650/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.523282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] RBAC: Authorizing `identity:create_grant(project_id=047614873c7a462ea30132f8c49e5842, group_id=664cc8922772431d980c7664fa956650, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.524746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.542899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f5aee-e8a3-4f67-8962-b0ff110163c9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.558890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:07.581611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.582626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.583088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.583541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.609396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 207/419] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Jun 7 09:08:07 2022] PUT /identity/v3/projects/047614873c7a462ea30132f8c49e5842/groups/664cc8922772431d980c7664fa956650/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:07.687327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.689605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6df3a-d2cd-4271-89a5-34d092b428eb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.817791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 213/420] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:07.823465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:07.953401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4860-673b-41a1-86b6-6f06431ad0cd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:07.955257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad37e11-aa65-4b81-941c-731872f862e6 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.955565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad37e11-aa65-4b81-941c-731872f862e6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.955826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad37e11-aa65-4b81-941c-731872f862e6 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.957487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 214/421] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:07.963805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74720-b18f-478f-a5ef-ee369c9248e2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.964090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74720-b18f-478f-a5ef-ee369c9248e2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.964363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74720-b18f-478f-a5ef-ee369c9248e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.983920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7de74720-b18f-478f-a5ef-ee369c9248e2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:07.985320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7de74720-b18f-478f-a5ef-ee369c9248e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:07.989700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:07.990566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:07.990852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:07.991128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:07.992346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:07.993839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:07.995846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 208/422] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:07 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:07.996149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.002976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.096888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.112208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 215/423] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:08.162449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946d9b-bdf5-492a-9cb0-56318d8cea4a None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.162770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946d9b-bdf5-492a-9cb0-56318d8cea4a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.163012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946d9b-bdf5-492a-9cb0-56318d8cea4a None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.165742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 216/424] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:08.169979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.170266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.170548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.175290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826498-c7b2-49d1-b099-c18d4660630f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.188971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:08.190286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:08.216029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.216834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.217171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.217451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.218611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:08.220058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:08.222611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 209/425] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:08.223384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.272928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 217/426] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:08.274559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.327155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209559-f046-4821-aea5-a230d9a60e02 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.345833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.366849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.367906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.368377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.368843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.381113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:08.382315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:08.390846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:08.395669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 210/427] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:08.396450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.418079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602314c-cec8-4410-955b-4a0b10955208 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.462810 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.463695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.464160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.464639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.465609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=529241aa434c450db6b72c5d75331f60) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:08.466203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:08.467785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:08.474918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 218/428] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:08.476034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.515487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7958cbd-3cb1-4228-8633-1b11026f4423 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.569973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.570791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.571071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.571340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.609660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:08.616857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc999-9c64-4e4f-929e-699a00c8a0dd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:08.618101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:08.618978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 09:08:08.619795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:08.629978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 211/429] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:08.642575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.666950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.668712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.669236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.669688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.670947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:08.672024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:08.678173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:08.692200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.696524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 219/430] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:08.704007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.708492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.815857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b54a-66ef-4579-8040-a35a9a8b904d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.831281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f8d78-0797-4cd4-ba29-83a08e2ab5f7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:08.868291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.868988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.869272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.869544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.870296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] RBAC: Adding query filter params (name=member) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:08.870599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:08.871976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:08.875227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:08.876130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:08.877209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:08.877705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:08.884341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 212/431] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:08.890454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:08.953880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:08.957325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:08.958998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.018913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.025896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d07c6-9334-4fc3-a1c6-32e85aac2819 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.064609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.065425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.065796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.066309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.070233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 220/432] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:09.075388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.139590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.141027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.181193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.184923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0d09-5953-427e-a89a-d60a20618bb7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.221364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.222461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.222889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.223330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] PATH_INFO: `/v3/projects/529241aa434c450db6b72c5d75331f60` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.224711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] RBAC: Authorizing `identity:get_project(project_id=529241aa434c450db6b72c5d75331f60)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.226298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.228486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 221/433] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/projects/529241aa434c450db6b72c5d75331f60 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:09.233430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.239965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.246898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 213/434] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:09.252280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.361430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586210d7-983e-41a1-accd-18296c864ac7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.369295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c2c3e-4210-4487-82e1-8b25c08a91f7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.402612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.403887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.404370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.404921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.406325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.408700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.411176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 214/435] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:09.411816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.416628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.416628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.416628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.416628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.416628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.417771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 222/436] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:09.418128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.423604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.459858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.508979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.527877 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba19e5-fe13-4332-8a27-db0b3607377d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.529720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfab5-b2ff-4b09-b8b1-e7122d2ed327 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.567775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.568018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.568661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.568953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.569135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.569255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.569652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.570060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.570476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.571176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=529241aa434c450db6b72c5d75331f60) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:09.571634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.571912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.573407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f453a99-a7ea-43ae-b933-4ab95cf05e1d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.573870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 215/437] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:09.580281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:09.582086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 223/438] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=529241aa434c450db6b72c5d75331f60&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:09.706777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3bbfd-53b0-44a5-9ba6-fab0dd1c79ce None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:09.739435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.744607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:09.745280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.745564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.748815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.748815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=da277faf5b1141e1aaa9bdaf1713ae18, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:09.748815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:09.748815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83665b79-9ae7-46f0-bec4-1164250304a1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:09.756035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 216/439] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:09 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=da277faf5b1141e1aaa9bdaf1713ae18 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:09.980451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acdb8e-a09c-4dcd-a92b-542890b99230 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.980840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acdb8e-a09c-4dcd-a92b-542890b99230 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.980963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acdb8e-a09c-4dcd-a92b-542890b99230 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:09.982467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 224/440] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:09.988195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7df71-4348-4f16-872e-704ad3f72467 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:09.988492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7df71-4348-4f16-872e-704ad3f72467 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:09.988830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7df71-4348-4f16-872e-704ad3f72467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.006570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f6d7df71-4348-4f16-872e-704ad3f72467 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:10.007947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f6d7df71-4348-4f16-872e-704ad3f72467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:10.078148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 217/441] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:10.101411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acdb8e-a09c-4dcd-a92b-542890b99230 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.209141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.210430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4353-9854-4dd6-9b19-c9418e75c5db None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.210647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4353-9854-4dd6-9b19-c9418e75c5db None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.210926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4353-9854-4dd6-9b19-c9418e75c5db None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.212624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 218/442] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:10.212944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acdb8e-a09c-4dcd-a92b-542890b99230 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.217445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.217757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.218019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.235717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:10.236892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:10.248189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.249021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.249300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.249570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.260028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:10.268938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:10.269992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:10.270924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 225/443] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:10.275709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.308835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 219/444] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:10.333298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.378331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335a32-044a-49e5-a386-564a12366fdb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.391851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.411959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.412640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.412926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.413200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.413938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:10.414243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:10.415603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:10.423492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 226/445] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:10.429191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.434541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b6bcf-fefe-4978-bd95-1cc9fe7b5107 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.452996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.468213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.469037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.469320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.469596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.480395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:10.488258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:10.489185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:10.490195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 220/446] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:10.495013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.528715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0751047-f575-4c62-a91a-5770fe97b4f7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.561837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.562476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.562758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.563034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.594288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40dd35-f518-4d5f-addb-5eef30c5f980 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.609962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.627128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.627780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.628060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.628334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.629120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:10.629430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:10.629693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:10.630802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:10.631130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:10.639503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 221/447] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:10.645335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.657566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.731881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 227/448] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:10.737505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.744240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779ca7-f374-4c0d-aa12-867ff1d2f022 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.778337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.778977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.779258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.779531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.823865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.836526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502e49e-6f77-4090-8bf7-488391af47ce None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:10.845174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:10.847102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:10.859650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:10.871916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:10.872931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:10.873219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:10.873494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] PATH_INFO: `/v3/projects/5d4e544e9f0c4114b0854e1efce46c91` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:10.874683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4e544e9f0c4114b0854e1efce46c91)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:10.876126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:10.877981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 228/449] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/projects/5d4e544e9f0c4114b0854e1efce46c91 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:10.883467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.946840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 222/450] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:10.952344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:10.982954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672372-9c3d-4605-94bd-20bf2af11f59 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:11.017573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:11.020800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:11.020800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:11.020800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] PATH_INFO: `/v3/groups/109cfde66d214ed097ebf6afaa269f96` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:11.020800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] RBAC: Authorizing `identity:get_group(group_id=109cfde66d214ed097ebf6afaa269f96)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:11.021598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:11.023219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 229/451] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/groups/109cfde66d214ed097ebf6afaa269f96 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:11.028876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:11.032306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:11.058066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3f54f-175a-4884-beb1-3159f920ef73 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:11.086095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:11.092795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:11.093644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:11.093925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:11.094199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] PATH_INFO: `/v3/projects/047614873c7a462ea30132f8c49e5842` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:11.096057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] RBAC: Authorizing `identity:get_project(project_id=047614873c7a462ea30132f8c49e5842)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:11.097538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:11.099231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 223/452] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:08:10 2022] GET /identity/v3/projects/047614873c7a462ea30132f8c49e5842 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:11.104658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:11.128929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c149354-eaf8-4f3d-a818-bebee16d55fa None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:11.162258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:11.162907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:11.163194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:11.163471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:11.164297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] RBAC: Adding query filter params (group.id=109cfde66d214ed097ebf6afaa269f96, role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=5d4e544e9f0c4114b0854e1efce46c91) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:11.164639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:11.166060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:11.171897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 230/453] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:11 2022] GET /identity/v3/role_assignments?group.id=109cfde66d214ed097ebf6afaa269f96&scope.project.id=5d4e544e9f0c4114b0854e1efce46c91&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:11.203715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daac6b3-91bc-4be7-959a-8b111f2a139e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:11.240185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:11.241111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:11.241388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:11.241656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] PATH_INFO: `/v3/groups/664cc8922772431d980c7664fa956650` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:11.242931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] RBAC: Authorizing `identity:get_group(group_id=664cc8922772431d980c7664fa956650)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:11.244412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11930d0b-b134-498b-9797-06f1590dfab7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:11.246326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 224/454] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Jun 7 09:08:11 2022] GET /identity/v3/groups/664cc8922772431d980c7664fa956650 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:11.252049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:11.355131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd048-aeac-481e-abf2-975497d94fa3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:11.377404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:11.392537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:11.393219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:11.393535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:11.402749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:11.402749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] RBAC: Adding query filter params (group.id=664cc8922772431d980c7664fa956650, role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=047614873c7a462ea30132f8c49e5842) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:11.402749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:11.402749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31876b7-0e1c-4d57-9ff9-342ca130d99c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:11.402749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 231/455] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Jun 7 09:08:11 2022] GET /identity/v3/role_assignments?group.id=664cc8922772431d980c7664fa956650&scope.project.id=047614873c7a462ea30132f8c49e5842&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:13.997830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55575-bacb-48c2-8264-e9df3d13f5f6 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:13.998395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55575-bacb-48c2-8264-e9df3d13f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:13.998395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55575-bacb-48c2-8264-e9df3d13f5f6 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:13.999719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 225/456] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:14.008665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548635-2caa-4084-a4ef-d99a355099e6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.009042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548635-2caa-4084-a4ef-d99a355099e6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.009219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548635-2caa-4084-a4ef-d99a355099e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.036651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-31548635-2caa-4084-a4ef-d99a355099e6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:14.038211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-31548635-2caa-4084-a4ef-d99a355099e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:14.130174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 232/457] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:14.178087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c55575-bacb-48c2-8264-e9df3d13f5f6 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.196298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:14.256870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d796220-1b6a-47d4-8286-d3d79f38d7c0 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.257238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d796220-1b6a-47d4-8286-d3d79f38d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.257751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d796220-1b6a-47d4-8286-d3d79f38d7c0 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.259554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 233/458] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:14.264939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d2cd-1d50-4e64-9b7b-3aa27335031f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.265457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d2cd-1d50-4e64-9b7b-3aa27335031f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.265929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d2cd-1d50-4e64-9b7b-3aa27335031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.300803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f845d2cd-1d50-4e64-9b7b-3aa27335031f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:14.302001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f845d2cd-1d50-4e64-9b7b-3aa27335031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:14.346034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c55575-bacb-48c2-8264-e9df3d13f5f6 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:14.394797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:14.395862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.396302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.396845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.404685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 234/459] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:14.406204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7fdb4-5671-475e-904c-95255a508742 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.406503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7fdb4-5671-475e-904c-95255a508742 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.406787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7fdb4-5671-475e-904c-95255a508742 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.408280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 235/460] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:14.413261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:14.413829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.414123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.414404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.415497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:14.436524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:14.437467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:14.438303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:14.438303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:14.438303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.438303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:14.440646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 226/461] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:14.441595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.443011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:14.444243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:14.496465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:14.528424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 236/462] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:14.529551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.543929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:14.595940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0f98-8265-44c3-af33-43164fa3c72b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:14.631283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:14.632314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.632858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.633126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.642922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88a88-ebd1-4367-8219-14a3110e69d3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:14.644769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:14.646221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:14.664933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:14.665813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:14.666655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:14.666655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:14.666655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.666655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:14.670955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 227/463] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:14.672083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.682581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:14.683676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.684187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.684797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.685718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:14.686237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:14.687946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:14.709556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 237/464] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:14.710933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.761401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:14.782865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fffef-6ade-4655-8be0-5578fea69878 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:14.802735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:14.821571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:14.823365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.823836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.824247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.835484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:14.837190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:14.847377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9989b2-a32c-4ffc-87d3-4b0235ba9ae5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:14.855611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:14.856457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:14.857297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:14.857297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:14.857297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:14.857297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:14.859443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 228/465] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:14.860424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.890627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:14.891595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:14.892742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:14.893283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:14.894353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:14.894854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:14.896635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:14.913921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 238/466] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:14.915231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:14.994998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a125ff-1369-46eb-ba63-237b210473e7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.031597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:15.046675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5ee9-5245-4df1-953e-ff8ed91ef01e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.080038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.080749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.081036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.081314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.082049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:15.082358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.083763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.097770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 239/467] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:15.099061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:15.163446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.164090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.164449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.164654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.165358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.166787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca042-eb38-4fa7-955d-cd835dead744 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.207535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19e34-001d-4d81-841b-dbd7d1d012d1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.241026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 229/468] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:14 2022] POST /identity/v3/users => generated 246 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:15.242491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c71f-6e15-4992-9134-7d302727e6de None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.242816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c71f-6e15-4992-9134-7d302727e6de None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.243098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c71f-6e15-4992-9134-7d302727e6de None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.244565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 230/469] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:15.245495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c71f-6e15-4992-9134-7d302727e6de None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:15.250927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.251652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.251975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.252245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.253148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.254655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d66f3-a61e-4cd0-af9f-91d743ec3f51 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.281790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:15.342149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 240/470] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:15.346724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.347128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.347457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.387849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:15.390086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:15.412405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:15.422338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c71f-6e15-4992-9134-7d302727e6de None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.493057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.493731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.494015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.494315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.494999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.496444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2698c8-137d-4fbc-a7b9-868201ed6c77 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.534277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 241/471] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:15.566119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:15.613827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 231/472] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/users => generated 248 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:15.674195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afb65-2cd8-45e5-8479-c4f8b8fc5435 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.674511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afb65-2cd8-45e5-8479-c4f8b8fc5435 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.674763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afb65-2cd8-45e5-8479-c4f8b8fc5435 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.676500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 232/473] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:15.682445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.682747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.683013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.696205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd89d0d-85e3-4a34-ac67-4e9bfde12cec None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.713767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:15.715373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:15.744714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.745596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.745823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.746098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.758729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.760249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.762551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:15.781616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:15.782733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:15.783584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:15.783584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:15.783584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:15.783584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:15.785840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 242/474] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:15.792418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:15.803005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 233/475] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:15.830874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:15.859900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:15.913033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240b77-2c54-4a19-9b4f-1bd2d92357fc None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.954863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a034e3-cb21-4a48-a872-ce848bb4b340 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:15.960412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:15.961114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:15.961398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:15.961674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:15.962406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:15.962715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:15.964145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:15.980298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 243/476] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:15.987255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:16.009361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3f403-106f-472e-908f-ba721047641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:16.010384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3f403-106f-472e-908f-ba721047641f None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:16.010863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3f403-106f-472e-908f-ba721047641f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:16.011666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3f403-106f-472e-908f-ba721047641f None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:16.012653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3f403-106f-472e-908f-ba721047641f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:16.014284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3f403-106f-472e-908f-ba721047641f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:16.051827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.sql.core [None req-44d3f403-106f-472e-908f-ba721047641f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 09:08:16.051827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 09:08:16.051827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [parameters: {'id': '3e9704f1497a4157b164fa7094865774', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 09:08:16.051827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80906) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 09:08:16.064850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-44d3f403-106f-472e-908f-ba721047641f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result.read() Jun 07 09:08:16.077465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:08:16.078366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application yield resource Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise self.value Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application session.commit() Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 09:08:16.079793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.session.flush() Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application compat.raise_( Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 09:08:16.080685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:08:16.085360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result.read() Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:08:16.086268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [parameters: {'id': '3e9704f1497a4157b164fa7094865774', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:16.087132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 09:08:16.087932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.089793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 234/477] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:16.089793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3f403-106f-472e-908f-ba721047641f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:16.128277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741400a-e565-4cca-8fb6-695c01b35ddd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:16.184124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:16.186011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:16.187135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:16.187418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:16.188132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:16.190244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:16.213633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:16.238506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3f403-106f-472e-908f-ba721047641f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:16.270057 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 244/478] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:15 2022] POST /identity/v3/users => generated 249 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:16.293444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:16.294263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:16.294669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:16.294947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:16.313941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-094f4495-06ce-4e7a-944c-d745c969f82b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:16.322729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:16.323698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:16.324735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 235/479] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:16.329865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:16.444397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3760281-7791-4073-9fb8-1feddd14ccb3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:16.456821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:16.498271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:16.498958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:16.499245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:16.499522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:16.500345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:16.500699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:16.502638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719259d-7012-457b-9f34-a6cf498e8b60 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:16.512426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 245/480] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:17.939078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e79651-da0e-431d-ab9c-ea3ad5d27ca1 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:17.939632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e79651-da0e-431d-ab9c-ea3ad5d27ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:17.939632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e79651-da0e-431d-ab9c-ea3ad5d27ca1 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:17.941350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 236/481] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:17.946709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:17.947150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:17.947367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:17.965214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:17.966458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:18.075744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 246/482] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:18.119176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e79651-da0e-431d-ab9c-ea3ad5d27ca1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.270315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e79651-da0e-431d-ab9c-ea3ad5d27ca1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.318724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.319521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.319801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.320076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.322837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:18.335465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f7110671-e9fa-4d05-b4b5-8774dca95da4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:18.343060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:18.343929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:18.344749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 237/483] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:18.353704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.449586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9dcd-f09b-4ae4-89cb-00afc8d2f17c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.449873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9dcd-f09b-4ae4-89cb-00afc8d2f17c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.450155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9dcd-f09b-4ae4-89cb-00afc8d2f17c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.452631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 238/484] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:18.456974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.457263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.457544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.492151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:18.492151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:18.495311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a67cb-db3f-47dd-9a8b-da6a56f8885e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.512619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:18.554774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.555685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.556108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.556638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.557623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:18.558064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:18.559581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:18.562047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 239/485] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 09:08:18.568572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 247/486] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:18.575384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.585478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.724764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e384-108e-45da-9bf7-fb8d3f4beda0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.729538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5589f8-698b-4598-a61c-2ef95a6a0c12 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.756940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:18.762507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.763673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.768392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.768392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.768551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.768551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.768551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.768551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.777543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:18.777712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:18.779021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c9951581-397f-432c-8d5f-532bb2061f11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:18.792801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:18.793795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:18.794585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:18.796688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 248/487] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:18.798680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346b0a-9ae8-4c86-a8ce-9d748f5914f2 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.799176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346b0a-9ae8-4c86-a8ce-9d748f5914f2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.799632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346b0a-9ae8-4c86-a8ce-9d748f5914f2 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:18.799821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:18.800634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:18.801461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:18.801461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:18.801461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:18.801461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:18.803670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 249/489] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:18.804545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346b0a-9ae8-4c86-a8ce-9d748f5914f2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.805771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 240/489] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:18.808104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:18.838872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:18.940548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958767a4-93c6-48ef-b7e0-114f4d1d2513 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.942381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346b0a-9ae8-4c86-a8ce-9d748f5914f2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:18.977999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.979489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.980019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.980523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.981535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:18.982046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:18.983665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9a2b8-d9a6-4428-8cc9-3bd5d712d9b0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:18.983771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:18.984452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.984828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.985202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:18.986742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:18.987115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:18.989022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:18.994946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 250/490] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:18.996614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:18.997109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:18.997563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.007995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 241/491] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:19.007995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.020348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:19.021660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:19.064284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.116947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 251/492] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:19.118186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.137186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e99e4-f72a-4963-a803-001672dea0e3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.148713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.179356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.180640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.182996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.183509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.197347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.198798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:19.217673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:19.218741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 242/493] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:19.219611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.251931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf77e-5efc-488b-9bf1-fbf0ee09a61e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.295602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.296442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.296865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.297141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.305601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.307059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.310651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.330707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b99fa-4c12-42ff-bfe8-26f573e8b5af None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-b5194d96-a87e-4560-ac9a-fad9de6b12bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:19.330870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:19.331719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 252/494] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401699cd-dc95-41ac-b5d6-6e5f8c184f91 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.332676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401699cd-dc95-41ac-b5d6-6e5f8c184f91 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.333364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401699cd-dc95-41ac-b5d6-6e5f8c184f91 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.333364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 253/495] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:19.333364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401699cd-dc95-41ac-b5d6-6e5f8c184f91 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.364431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.365346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.365636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.365919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.376135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:19.383822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:19.384784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:19.385582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 243/496] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:19.386565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.413339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.436711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401699cd-dc95-41ac-b5d6-6e5f8c184f91 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.471602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.472254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.472540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.472932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.473664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:19.473974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.475357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bb3c4-1065-485c-b145-118a19b6c7f7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.484773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6937d-1733-43da-8b19-45c83a17789b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.486514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 254/497] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:19.487694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.487962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.488234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.504342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:19.505587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:19.519660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.520332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.520605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.520900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.521729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:19.522034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.523442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16ba8b-2e51-45b1-9ed7-9b6fccf2569e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.558680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 244/498] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:19.559935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575167-3129-4bad-9824-21c65fd18992 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.560220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575167-3129-4bad-9824-21c65fd18992 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.560494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575167-3129-4bad-9824-21c65fd18992 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.561729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 245/499] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:19.562405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b575167-3129-4bad-9824-21c65fd18992 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.572993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.577554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 255/500] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:19.578413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.643910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.676024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b575167-3129-4bad-9824-21c65fd18992 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.676279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cee93-6ea2-4408-ab47-6c7202cae86b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.710020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.710676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.710869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.711004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.711212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.711679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.711939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.712027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:19.712208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.712296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.713617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.720114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.721437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 246/501] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:19.721571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.722208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2e771cac-50ea-4326-a1e7-4a7f25d8e0f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:19.740512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:19.741487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:19.742385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 256/502] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:19.742955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.743477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.743477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.759087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:19.760263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:19.792938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.823195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e077b1-2ba2-4e79-862c-ea70ac294e88 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.825504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 257/503] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:19.826330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.858220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.858862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.859145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.859421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.882466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:19.925280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84060-1eb5-4590-b9d7-253fbfb7c5cc None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:19.927502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:19.928977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:19.958644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:19.959434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:19.959718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:19.959997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:19.970411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:19.978088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:19.979407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 258/504] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:19.980189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:19.999617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.031067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 247/505] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:20.031808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.082940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8561-75c6-4410-aeff-4a7b32169361 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.099100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.117129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.117916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.118258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.118538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.129460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.129978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377409b-0a11-47fe-b7b4-9141d93d4bb0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.130997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:20.148449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:20.149347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 259/506] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:20.154984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.163501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.164136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.164413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.164711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.165541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:20.165842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.167240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.184727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 248/507] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:20.185468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.212639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.254378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebecc93-f81e-4856-a429-169e0ccbfd79 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.283542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f6cc5-b910-40aa-af90-a638db08c863 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.287232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.288013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.288289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.288561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.299688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.301735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.317986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.318642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:20.318902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:20.319875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:20.320982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:20.320982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:20.320982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.320982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:20.321190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.321190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.321190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:20.321190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.321429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 260/508] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:20.321429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.321688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.330800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 249/509] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:20.331490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.345455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.421640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397eb8-54b7-4cf7-b5b6-98bfbf19afb3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.421812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.430975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479e30-46dc-4237-a18b-7d48ef2afc6f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.454850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.455568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.455850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.456126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.464592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.465264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.465547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.465824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.466515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.466605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:20.466873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.467970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.468219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.479968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 250/510] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:20.480777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:20.485580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:20.486589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 261/511] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:20.491762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.558006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.580873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6e7e-7dbf-48ca-827d-3f79158d3d66 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.585916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2654c-09d0-4fa9-8678-94c41a7222e9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.613794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.614426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.614711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.614973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.627385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.628039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.628325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.628632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.629475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:20.629783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.631158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.633195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.643406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 262/512] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:20.644220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.682355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.683798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.745646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba83031-8209-4160-b713-01d4ebd53d6d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.774933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.779767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.780549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.780669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 251/513] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:20.780865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.781137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.781483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.795617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.797230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:20.816175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:20.817093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:20.817929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:20.817929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:20.817929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:20.817929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:20.817929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 263/514] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:20.818629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:20.832056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:20.882232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87565b-a408-4bf4-a273-cb605f279f36 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:20.918208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:20.919019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:20.919301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:20.919578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:20.928698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:20.930150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:20.946585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:20.947515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:20.948410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 252/515] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:20.949478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.054787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5a312-c914-440f-b2d5-b07fbfe3058b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.062548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b12d5-582d-4409-bf52-04f0b0fc6e22 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.065265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.088627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.089297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.089578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.089855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.090593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.092898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.096335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.097028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.097309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.097582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.098298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:21.098606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.099969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.112041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 264/516] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:21.112839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.157998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.177309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 253/517] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:20 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:21.178131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.215603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03924-280e-4a99-9fde-009389511cbd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.257205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.257922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.258208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.258487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.270709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.272199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:21.293163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:21.294097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:21.294941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:21.294941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:21.294941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:21.294941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:21.294941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 265/518] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:21.295211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.321906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2781083-ddd6-488c-9c4e-d1df23953af0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.357669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.358351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.358637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.358911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.359654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:21.359961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.361909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.374867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 254/519] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:21.375709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.402646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.408699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.410771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cbddc-801b-442f-86a6-eb640abc1686 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.447908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.448606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.448932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.449204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.450035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:21.450338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.451761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.488156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 266/520] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:21.489026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.498378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b591-ae76-48d0-b7d6-caeff09a16e7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.536000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.536750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.537036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.537306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.544144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.545668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:21.560462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:21.561378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:21.562231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 255/521] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:21.562716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.598658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50771-a0ec-4928-a48e-f854f6299b67 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.633350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.634149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.634536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.634906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.635735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:21.636312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.637716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.649264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 267/522] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:21.650072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.654330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.666644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8678c23-fd68-44f1-9dd2-1d837e1ab467 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.672335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.701875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.702672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.702954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.703225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.710529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.711966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:21.727259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:21.728176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:21.729087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 256/523] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:21.730276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.767439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092ddc-632b-444a-a5e9-208632aa5300 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.803419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.803991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.804278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.804555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.836573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72f60-abe5-4cce-a105-5a5ef0c11b94 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:21.873147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:21.873804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:21.874086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:21.874360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:21.875156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:21.875465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.876980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.877689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:21.879207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:21.895542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.903107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 257/524] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:21.903107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:21.911995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:21.983369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 268/525] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:21.984312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.024071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a80ff-6794-4d89-9772-baa015b05483 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.061557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.062415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.062828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.063120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.070424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.071875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:22.086913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:22.087738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:22.088628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 258/526] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:22.089097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.100193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355aec4-2165-43d1-8765-84671a18326d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.117579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.138239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.138915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.139196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.139473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.140362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:22.140701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.142763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.161539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 269/527] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Jun 7 09:08:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:22.162827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.166712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.198704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f327c-b3f3-42b6-a233-6a05f7b03c96 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.234240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.234883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.235169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.235552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.236381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:22.236722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.238136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.243860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 259/528] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:22.244448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.280163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660190f4-9a7d-4f9f-a384-7d7a1e96277d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.325069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.326257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.326856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.327339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.342421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.344992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:22.372931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:22.373867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:22.374705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:22.374705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:22.374705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:22.374705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:22.382117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 270/529] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:22.382117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.388224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900a4a-2c1f-41d3-86bb-5bea10b7bc57 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.411700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.424913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.432009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.432932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.433286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.433634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.434513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:22.435162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.436423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.452058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 260/530] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:22.453142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.507519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40d659-6ae8-40dc-9481-aa6dbb1a1894 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.545016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.545673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.545957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.546234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.567871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db69fd-f5cc-48a8-921a-a1ba1823cffc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.605078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.606174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.606637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.607087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.608147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:22.608747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.610849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.619127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.620737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.628662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 261/531] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:22.629755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.647503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.683995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.735031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 271/532] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:22.736734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.739783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17062-71bb-4caf-a42c-996ff8610acf None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.777777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.778723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.779269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.779688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.780718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=7025724d5c31460db223411aab77003a) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:22.781190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.782708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.796612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 262/533] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/role_assignments?user.id=7025724d5c31460db223411aab77003a&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:22.796612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:22.899637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c0bf8-0d99-47fe-9de2-449e442e4496 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:22.947390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:22.971849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:22.972677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:22.972959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:22.973280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:22.985735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:22.987227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:22.993369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf7e3-8e0e-4561-b764-f4df5cd9a353 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:23.016095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:23.019775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:23.020790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 272/534] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:23.024462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.030230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:23.057956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.059813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.060327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.060712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.079706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.079706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:23.099522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:23.100609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:23.101665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:23.101665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:23.101665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:23.101665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:23.103114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 263/535] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:23.108371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.181558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765661-e0fa-4ed0-ab2d-22a1b747136a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.217345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.218009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.218293 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.218569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.219363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:23.220212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.221745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.244702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 273/536] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:23.246414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.297434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:23.354716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172bc0d-c1f5-4f18-a42f-130a087e124d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.355999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a4597-e0ea-4878-bd76-7233cb6d54f3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.393393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.394056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.394394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.394677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.395510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=c7d851c1cb8b495a87692b8e8970f851) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:23.395821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.397355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.404359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 274/537] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/role_assignments?user.id=c7d851c1cb8b495a87692b8e8970f851&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:23.434405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.435001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.435286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.435558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.436303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:23.436626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.438054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c817bd2-d248-4be0-94be-e062409148ee None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.458450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 264/538] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:23.464080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.535403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774addd7-db7c-4889-a008-e942530bed5c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.536059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774addd7-db7c-4889-a008-e942530bed5c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.536554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774addd7-db7c-4889-a008-e942530bed5c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.539686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 265/539] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:23.544466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.545082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.545552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.572285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:23.573557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:23.621169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:23.652071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d2ec-75e7-4974-ac48-4b13fe3d2d93 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.674603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:23.709500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 266/540] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:23.713469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.714336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.714619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.714895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.722957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.724407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:23.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:23.739238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:23.740744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 275/541] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:23.745185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.756142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.868661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3eb63d-c922-4949-bda4-cb7cf08fd58f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.921517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.922185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.922471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.922809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.923603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:23.923915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:23.925379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:23.941287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5678628-0d62-4aeb-a8fd-65e7fa006922 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:23.944753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 267/542] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:23.950527 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:23.975583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:23.976309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:23.976612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:23.976899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:23.988160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:23.994375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:23.997298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:23.998202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:23.998996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:24.001918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 276/543] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:24.011472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.062753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:24.146727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e02e9-cb28-421c-8883-ec0e38a7fb9e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.183497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.184255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.184438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.184752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.185502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:24.185805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.187175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.201502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 277/544] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:24.202797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.234629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da545e-8818-4121-8530-6f0cc16514d0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.301022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:24.310430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ca2fe-9c65-4e1d-84c8-91326f66fe1a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.352604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.352604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.352604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.352604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.353768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.354595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.355104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.355545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.356533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=943602b745284e6b9a53ff217a9ee0b4) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:24.357013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.358616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.368603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.368603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.373417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 268/545] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:23 2022] GET /identity/v3/role_assignments?user.id=943602b745284e6b9a53ff217a9ee0b4&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:24.383128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:24.384060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:24.384923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:24.384923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:24.384923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:24.384923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:24.388820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 278/546] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:24.390811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.504724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:24.516248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbd229-ee73-4862-bb16-3ccd29db97f2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.561699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.562372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.562810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.562937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.564191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:24.564497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.565903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.583276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 269/547] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:24.584607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.689990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e44d0a-5f11-4acb-922c-8a34f5662ff8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.724323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.725181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.725462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.725732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.733498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.734940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-fe8bdb3c-7bf5-4903-af7a-ed9a86527068 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:24.749176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:24.750180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:24.751689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:24.753808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 279/548] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:24.757002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.862420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba162-3d20-4b8d-858c-712da4af0ec1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:24.897374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:24.898049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.898333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.898608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.899447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:24.899753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:24.901202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:24.911311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfff3d-f520-4fb6-8112-16f5637b5642 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.911613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfff3d-f520-4fb6-8112-16f5637b5642 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.911863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfff3d-f520-4fb6-8112-16f5637b5642 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.913524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 280/549] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:24.918659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:24.919004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:24.919191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:24.921868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:24.925216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 270/550] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:24.928849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:24.937269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:24.938610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:24.940897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.022072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 281/551] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:25.075450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.089838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1dd1-3778-4e9e-ac5c-f0e1990a9034 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.149946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.150606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.150890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.151165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.193003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78adcc-be43-4594-97f5-fc0ce398c011 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.228983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.234034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.235044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.238596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.238596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.250397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:25.261404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:25.262566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:25.263693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:25.266188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 282/552] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:25.275407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.297250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.308442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:25.314524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:25.390430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f256dc-034b-4d95-8abc-989c92225217 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.428129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.428827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.429109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.429382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.430127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:25.430429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:25.431993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:25.442024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 283/553] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:25.449597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.538905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 271/554] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 09:08:25.543848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.561350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.604015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd482143-98ff-4691-8eb9-39bbd9c56645 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.644303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.645635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.646215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.646864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.659271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:25.663387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:25.680614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d6c74e5d-dc2e-448a-97a4-a7930e78e5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:25.682812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:25.683797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:25.684870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:25.684870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:25.684870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:25.684870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:25.685182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632eb26-6d3b-4800-8da9-c50bccb48eab None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.691925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 284/555] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:25.693757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be025b2-eec4-41b3-9bae-999ec1208763 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.697842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be025b2-eec4-41b3-9bae-999ec1208763 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.697842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be025b2-eec4-41b3-9bae-999ec1208763 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.697842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 285/556] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:25.697842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be025b2-eec4-41b3-9bae-999ec1208763 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.732143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.733237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.733678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.734127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.745901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:25.747605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-7bf1b552-e850-4f4b-bf30-06a60e590706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:25.767424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:25.768548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:25.770183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:25.770183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:25.770183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:25.770183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:25.772052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 272/557] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:25.773568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.773996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.774410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.791719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:25.793237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:25.827913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be025b2-eec4-41b3-9bae-999ec1208763 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:25.836513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.865923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 273/558] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:25.867048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.885274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:25.886417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:25.886862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:25.887280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:25.889121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:25.890439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:25.892707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:25.906863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 286/559] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:25.912175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:25.968721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:25.989431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23aa30d-4aeb-4a7e-9405-dd7652a4b85c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.029244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.029911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.030239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.030459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.031194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:26.031496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.033619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.041677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41acfd7-f65b-4fb3-b48a-443b910b3e54 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.049906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 274/560] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:26.049906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.082548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.083348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.083623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.083898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.094564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-cc91d942-5d1a-4910-a986-c9f063c891a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:26.102261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.103218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:26.104041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 287/561] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.105536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661ba-7825-4548-8523-6b672950707b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.105536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661ba-7825-4548-8523-6b672950707b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.105536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661ba-7825-4548-8523-6b672950707b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.106679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 288/562] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:26.107196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424661ba-7825-4548-8523-6b672950707b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.136716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.164763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6c032-5556-41fe-b47b-a230bc17946d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.203651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.205024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.205348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.205714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.213262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.214651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424661ba-7825-4548-8523-6b672950707b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.215109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.218153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:26.229408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.230555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:26.231516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 275/563] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.232096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.257188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.257978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.258260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.258532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.266205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.268190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c742376c-b198-4aa1-8d0f-702a4d8144e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:26.282217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.283143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:26.284041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 289/564] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.285026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.285137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.285427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.301870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:26.303007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:26.344462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d004151-8dad-4c35-afc3-670df4c7ee80 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.374183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 290/565] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:26.375314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.392210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.393582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.393862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.394939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.395420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.396393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:26.396963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.398331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.406788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 276/566] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:26.407626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.471817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.489154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f02473-0a45-407d-a4c9-169fc909fde8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.513249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecfbd1-c630-4712-a046-ca351f532297 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.524821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.525493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.525775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.526046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.526874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:26.527187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.528716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.546611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 291/567] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:26.547438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.555198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.556079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.556689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.556971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.557769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:26.558077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.559618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.577681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 277/568] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:26.577681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.627858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.653971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f76344-4e78-43c1-8d1c-69ecab8ede2c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.682235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa03130-4198-475c-aafd-9b21b8cbaa10 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.690065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.690866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.691146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.691420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.701532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:26.708931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:26.709215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.710149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:26.710976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 292/569] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.711683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.724341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.725405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.726003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.726355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.738635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.740085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:26.760562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.761820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.762689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:26.762689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:26.762689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:26.762689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:26.762689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 278/570] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.762971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.823111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9b3e0-b080-48e8-96f7-e64b9252c83d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.858780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.859571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.859851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.860220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.862288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:26.871232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.872717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.877080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c5cb-6a42-43c4-84c5-cc2a0b462bba None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:26.890055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:26.890972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:26.891933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:26.891933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:26.891933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:26.891933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:26.891933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 293/571] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:26.892226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.914217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:26.914871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:26.915148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:26.915478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:26.916311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=327a3409391e4d558aeab965401f48e0) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:26.916652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:26.918090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:26.924624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 279/572] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/role_assignments?user.id=327a3409391e4d558aeab965401f48e0&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:26.925359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:26.975896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.045122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320d7c-3b77-46fc-8820-0f402f3d6bc0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.061429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea862e4-f837-45d6-b778-0db29cd3f7db None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.087712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.088820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.089041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.089327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.090106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.090793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.092166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.101925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 294/573] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.102786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.106984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.107888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.108181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.108458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.109236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.109547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.111522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bceb88a-41e5-4e4f-90c6-9c890a536448 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.124527 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 280/574] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.126515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c550-7cc3-4a38-b464-2f03bd5958a1 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.126818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c550-7cc3-4a38-b464-2f03bd5958a1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.127440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c550-7cc3-4a38-b464-2f03bd5958a1 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.128615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 281/575] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:27.129042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94c550-7cc3-4a38-b464-2f03bd5958a1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.165820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.217708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fd54a-efd7-4c24-b123-b506d9d9d998 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.236888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.241554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94c550-7cc3-4a38-b464-2f03bd5958a1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.254134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.254795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.255076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.255347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.256069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.256366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.257775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.273329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 295/576] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.274335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.283376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.284173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.284463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.284773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.295774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.297256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-23d8db7e-7e09-4471-badd-8a0313855ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:27.315551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:27.316665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:27.317821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:27.317821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:27.317821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:27.317821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:27.321761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 282/577] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:27.322850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.323095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.323340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.340236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:27.341401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:27.381606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a2275-32f2-4bb4-be5c-29892bcc7d33 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.392931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.420651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 283/578] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:27.420651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.431364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.433086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.433623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.434082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.443312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.444800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:27.459430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:27.460573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:27.462080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 296/579] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:27.464148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.509220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.528720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e1d4e-1d3b-4505-95a9-f03971b7abaa None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.566596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.567391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.567676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.567942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.572889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178b847-566b-4754-997f-e1426dbe7b60 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.575368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.576852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:27.590731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:27.591606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:27.592481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:27.597062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 284/580] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:27.597913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.609478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.610283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.610634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.610927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.611643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.611962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.613357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.627120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 297/581] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.627906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.661254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.714881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0ce21-bdb7-4e98-8c76-2ceb35cb17f9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.737548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a961ba9-a243-47b5-91ad-16cc3d0a7a1d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.752392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.753333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.753710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.754082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.762695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.765824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:27.774289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.775012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.778186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:27.778283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:27.779171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:27.780021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:27.780700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.780700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.780700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.780700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.780955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 285/582] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:27.782482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.797653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 298/583] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.798426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.887285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:27.891919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2270a-2c65-41bf-92d6-9bdb64abf4be None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.903174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd4cdd-b17c-4d4a-b795-21469b313a49 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:27.926948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.927598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.927883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.928165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.929037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:27.929348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.930768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.939119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:27.939991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:27.940347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:27.940722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:27.947883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 286/584] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:27.947883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:27.955202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:27.957124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:27.972198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:27.973230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 299/585] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:27.974173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.010375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.059493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae9754-050c-4917-bf29-3dfcddae109a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.093982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362ff4f-e2f7-43ac-a355-f5c35873c090 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.099405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.100351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.100971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.101976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.102944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:28.103882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.106159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.116200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 287/586] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:28.117232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.131928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.132947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.133610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.133896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.148244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.149752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.156683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:28.168204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:28.172808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 300/587] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:28.173798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.233226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c4d54-1624-4368-832d-f5ddc2176bda None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.257346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.269386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.270277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.270560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.270840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.280684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4491a4-7876-48a1-83f2-c3a4665dc8be None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.287874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=dff6b630f6be464aa6cdcc73da587e4f, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.289341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.315525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.316180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.316459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.316760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.317579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:28.317882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.319292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.337195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 301/588] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:28.337966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.368619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 288/589] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:28 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:28.368619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.414859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.475535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d24e0-4991-4869-a4bf-d7b8998814fd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.499494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ad93d-d21f-41a2-b280-f8bca5731b57 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.515300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.515974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.516922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.517219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.530360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.532274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.552224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:28.552438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:28.553324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:28.554196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:28.554196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:28.554196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.554196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:28.555595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.556046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.556477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.557422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:28.557890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.558698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 302/590] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:28.559724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.559968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.575145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 289/591] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:28.575786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.620928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.697372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f005-5c30-4d12-972f-2007c2b9f6af None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.708364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ff9ad-2dcc-4659-b62a-7a1ed7322655 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.712278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.745533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.747319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.747825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.748299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.751737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.752759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.753051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.753332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.754068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:28.754380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.755749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.762605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.764995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.770528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 290/592] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:28.771445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:28.788683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:28.789630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:28.790520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:28.790520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:28.790520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:28.790520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:28.793856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 303/593] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:28.795238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.890555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:28.901615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a478a30-a9fc-4c90-8b78-6c277a4c488d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.937618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62a29-c110-4d32-b5c3-1f13c5cbbc14 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:28.940116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.940957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.941249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.941528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.953280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.956211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:28.975145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:28.975507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:28.976556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:28.977701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:28.978485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:28.978871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:28.979279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:28.980290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 291/594] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:28.981239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:28.991518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:28.992861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:29.008769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:29.009713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:29.010656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 304/595] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:29.011513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.040467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.116803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908d39-82a2-40c0-aeb7-6dde70f9ee2e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.156138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.157357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.157646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.157923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.158702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:29.158961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.160332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.161219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b1d77-ebb5-4470-852b-550ed951deb1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.172905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 292/596] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:29.173734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.190319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.206812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.207404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.207689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.207973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.208785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:29.209100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.210577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.227653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 305/597] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:29.227653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.285226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8d36-0468-44e8-8f18-ca1391d636b3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.310070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.323812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.324483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.324802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.325072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.325913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:29.326227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.327663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a0e7b-bfaf-44b2-b159-e0447fdfe75c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.330159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f01d-cb0f-4186-93a3-c6ae28c6752c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.347002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 293/598] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:29.348124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe3eaf-f3b9-442b-ad49-d1883bb56aa4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.348420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe3eaf-f3b9-442b-ad49-d1883bb56aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.348726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe3eaf-f3b9-442b-ad49-d1883bb56aa4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.354624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 294/599] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:29.354624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe3eaf-f3b9-442b-ad49-d1883bb56aa4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.374444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.375231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.375528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.375914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.382572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.384023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:29.398607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:29.399546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:29.400438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:29.404541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 306/600] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:29.405384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.429733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.462147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe3eaf-f3b9-442b-ad49-d1883bb56aa4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.499212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.500213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.500869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.500951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/7025724d5c31460db223411aab77003a/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.507607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0d0-b939-4bb0-942b-026791a12bd9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.515941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=7025724d5c31460db223411aab77003a, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.518063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.541731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.542509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.542787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.543055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.551093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.553090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.554520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d9fbaf72-54df-4587-beaa-468a8bf99a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:29.571471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:29.572497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:29.573461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:29.573461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:29.573461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:29.573461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:29.573461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 307/601] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:29.574177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.574399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.574675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.594838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:29.596034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:29.604054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 295/602] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:29 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/7025724d5c31460db223411aab77003a/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:29.605444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.704754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 308/603] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:29.705722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.759171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6c14a-e536-434c-ba11-38367c61155a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.788761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.806966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.807911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.808195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.808474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.809457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:29.810106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.811836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.828023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 296/604] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:29.829035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.831068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11b215-8b8f-46a2-b3fe-c185219156f4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.855125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:29.867160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.868096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.868619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.869115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.870025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:29.870525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:29.872085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:29.884486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 309/605] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:29.885580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:29.935780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fe115-3aff-461c-b515-681c32fde6eb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:29.970563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:29.971738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:29.972280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:29.972798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:29.985910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:29.996740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:29.997697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:29.998538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 297/606] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:29.999237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:30.017299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df03fb-e546-45a8-b9b5-2067c9344926 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.072979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:30.085307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.087296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.087296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.087296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/c7d851c1cb8b495a87692b8e8970f851/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.128534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2190a9-49f3-4330-b203-b9c910876007 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.136617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=c7d851c1cb8b495a87692b8e8970f851, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.140605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.200609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.200609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.200609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.200609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.208402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.209945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:30.225696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:30.226639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:30.227607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 298/607] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:30.228091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:30.272688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 310/608] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:29 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/c7d851c1cb8b495a87692b8e8970f851/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:30.272688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:30.399424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:30.539334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:30.544009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12520d6e-7342-434b-8b00-a6440859ac13 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.583940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280a6f2-073d-4259-a8b7-9478ac89df4c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.595973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.597134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.597566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.597988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.598924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:30.599529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.601932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.611864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 299/609] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:30.617116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:30.656772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.659665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.660127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.660550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.661581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:30.664290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.668058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.693101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 311/610] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:30.701398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:30.825766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca6f8-74b8-4245-bb78-27d20873265e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.827879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:30.837704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b90734-bb92-4906-a72f-c900925a941c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:30.871812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.872671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.872888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.873181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.877288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:30.878356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.878778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.879190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/943602b745284e6b9a53ff217a9ee0b4/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.884689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.886419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.896760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=943602b745284e6b9a53ff217a9ee0b4, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:30.898543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-7b0256e3-c497-4919-881e-8a3560af7e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:30.905488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:30.906413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:30.907408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:30.907408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:30.907408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:30.907408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:30.907408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 300/611] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:30.908378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970f86c-9252-48c7-97ec-4455c30c7132 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.912659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970f86c-9252-48c7-97ec-4455c30c7132 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.912659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970f86c-9252-48c7-97ec-4455c30c7132 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.912659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 301/612] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:30.916461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:30.916780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:30.917101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:30.935051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:30.936452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:30.946599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:30.968179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 312/613] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:30 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/943602b745284e6b9a53ff217a9ee0b4/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:30.969721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.028633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 302/614] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:31.089868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.136923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2bce4-efe2-464f-9820-8db6bffc7d23 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.175547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.176613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.177099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.177569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.178523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:31.179025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:31.180665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:31.196961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 313/615] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:31.200652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.205883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:31.255870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890c53e-da74-42e8-a98f-2a51ccfecb94 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.304439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.305278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.305565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.305846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.330503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:31.331078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda3a6-ec99-48e4-a4b3-341844adbca0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:31.349701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:31.350635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:31.351450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:31.355368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 303/616] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:31.361244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.368194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:31.407637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.409148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.409923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.411706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.428963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:31.430718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:31.458168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:31.459345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:31.460294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:31.476680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 314/617] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:31.494098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.574154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:31.612842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5565-d4b3-4d8b-aa05-46958ad72f7a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.616117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07933e-60e5-4ff5-ae4f-543d423636d9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.659409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.661010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.661499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.662107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.663208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:31.663851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:31.665545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:31.692900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 315/618] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:31.695321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.713353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.714048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.714337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.714614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.715377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:31.715685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:31.717098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:31.750937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 304/619] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:31.757556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:31.806837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dee7c-d6cd-4147-a6fd-844322230b2d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:31.829669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:31.845116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:31.846694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:31.847076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:31.847413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:31.859173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:31.861571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:31.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:31.882069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:31.882971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:31.882971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:31.882971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:31.882971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:31.887102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 316/620] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:31.889272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.001300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151d14-02a4-44ca-9f8a-711b71b87b06 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.015569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.040489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.041183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.041466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.041742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.042466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:32.042774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.044401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d5e7-e687-4b20-9360-7462b14ddf03 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.060675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 317/621] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:32.062034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945b25-47fc-4628-91eb-eac1a7e46bc3 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.062320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945b25-47fc-4628-91eb-eac1a7e46bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.067158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945b25-47fc-4628-91eb-eac1a7e46bc3 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.067158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 318/622] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:32.070205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e945b25-47fc-4628-91eb-eac1a7e46bc3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.086820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbf475-c686-42a5-bba4-59c0971438ca None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.112945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.121282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.122068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.122345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.122616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.135416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.137083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-b800a5c6-9e99-4844-8889-02ba31f36a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:32.154524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:32.155429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:32.157475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:32.157475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:32.157475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:32.157475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:32.157475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 305/623] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:32.157765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.158020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.158293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.175156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:32.176326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:32.216878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e945b25-47fc-4628-91eb-eac1a7e46bc3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.252856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 306/624] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:32.253759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.261898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.262950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.263460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.263927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.272577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.274597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:32.293081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:32.294053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:32.294935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:32.303475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 319/625] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:32.303475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.342153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.404830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023982-32be-4c90-93f1-5255c80bfbec None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.410695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42fec-8eb1-4426-81d5-494fed69abc4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.413764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.443431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.444504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.444972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.445738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.446681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:32.446788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.447291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.447562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.447839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.448108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.449535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.459442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:32.467183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 307/626] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:32.468030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:32.468987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:32.469896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:32.470539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.476711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 320/627] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:32.476711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.600811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc36f7-3da3-4165-bf8f-059f59aee567 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.604055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e1cb-fdf9-4391-ab43-fc6f28514aaf None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.612994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.640460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.642131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.642612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.643141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.645585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.646901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.647188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.647463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.648251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:32.648557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.650003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.652801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.654489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.668320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 308/628] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:32.669147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:32.670070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:32.671006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:32.671530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.673933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 321/629] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:32.674887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.691256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.790689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf6050-0988-4834-9acf-294c512fe167 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.794741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75048199-0a28-448b-b8b1-14d10b530b7d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:32.832208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.832908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.833189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.833554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.834408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:32.834718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.835758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:32.836127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.837143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:32.837678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:32.838244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/327a3409391e4d558aeab965401f48e0/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:32.848667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 309/630] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:32.849821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.861009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=327a3409391e4d558aeab965401f48e0, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:32.862477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:32.883849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:32.937120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 322/631] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:32 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/327a3409391e4d558aeab965401f48e0/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:32.937703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:32.963539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ade1e-59c4-4180-ab5e-57cf7c45a324 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.006237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.007024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.007379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.007723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.008782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:33.009207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.011102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc4386-2f44-459f-af14-2a9bb181689a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.035030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.037262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 310/632] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:33.038805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818aa95-4dde-4cd4-8aa1-c1c07b8c1537 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.039100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818aa95-4dde-4cd4-8aa1-c1c07b8c1537 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.039377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818aa95-4dde-4cd4-8aa1-c1c07b8c1537 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.045064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 311/633] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:33.046229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818aa95-4dde-4cd4-8aa1-c1c07b8c1537 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.089502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f4214-a2dd-45fc-877a-cb33a23ae8d0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.130468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.131293 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.131670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.131844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.147881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.149426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2a2f28f8-5f01-414b-8212-24623321f7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:33.170134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:33.171101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:33.171984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:33.171984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:33.171984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:33.171984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:33.171984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 323/634] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:33.173459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.173738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.174060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.176970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.195667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:33.196809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:33.197221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818aa95-4dde-4cd4-8aa1-c1c07b8c1537 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.237809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.238395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.238725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.239424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.283455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 324/635] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:33.287868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.341131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.342732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.361937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.427088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d9794-4d81-4800-9a07-edef82455f9b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.460065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.464107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.464974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.465352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.465723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.466516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:33.473213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.473213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be312a75-ff9a-46b9-a522-72c539b886a1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.476677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 312/636] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:33.478089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.481409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 325/637] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:33.482883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483257-e41c-4011-8d6d-02289d236488 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.483168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483257-e41c-4011-8d6d-02289d236488 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.483440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483257-e41c-4011-8d6d-02289d236488 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.484851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 326/638] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:33.485453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe483257-e41c-4011-8d6d-02289d236488 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.592991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a55d77-f0a6-4ad1-9db0-bbe03940cc52 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.593946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe483257-e41c-4011-8d6d-02289d236488 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.597125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.628284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.629098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.629378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.629648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.635906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.636723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.637263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.637263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.641337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.642781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.650117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-286e7e6c-4a15-431e-9aef-3a0a6930af1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:33.659763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:33.660723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:33.661687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:33.667746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:33.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 327/639] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:33.669490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.669833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 313/640] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:33.669833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.669833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.669833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.685114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:33.686288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:33.731970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.755549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 314/641] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:33.756658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.785521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907791-9d65-4cee-affd-8a69acd9b49c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.823469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.824255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.824535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.824989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.832389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.833868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:33.848467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:33.849817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 09:08:33.851001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:33.854233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 328/642] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:33.855073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.868796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb51a9-ebf6-4424-b71f-daa1eb4aba59 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:33.874417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.906587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:33.907313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:33.907581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:33.907858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:33.908575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:33.908906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:33.910260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:33.926965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 315/643] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:33.926965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:33.965813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:33.969279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890815-2ce2-4ab5-910f-40857b591dbb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.007319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.007942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.008221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.008491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.009460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.009769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.011393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.022839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 329/644] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.027650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.036366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201933-fa12-47cc-ab70-8e38c2ad9a6c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.073849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.074782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.075164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.075541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.086681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:34.094399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:34.095518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:34.096521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 316/645] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:34.097288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.114905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.136912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9c89a-75bf-4ca3-9f3c-eb97500a46e0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.175546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.176536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.176853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.177124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.178155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.178269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.179725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.203157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 330/646] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.204427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.205831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.210565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7467a-2bae-4862-9c13-6ed2e1a99b62 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.246856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.248073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.248357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.248653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.256798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.258300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:34.272822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:34.273744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:34.274634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 317/647] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:34.275111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.314339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550e822-1a6a-4b3a-a392-1a4d93232148 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.349895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.350695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.350981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.351253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.354612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.363155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.364627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:34.383005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:34.384107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:34.385213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:34.385213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:34.385213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:34.385213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:34.389068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 331/648] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:34.389893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.390276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f7abd-278c-4454-8f16-13f80a5736e2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.428356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.429332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.429615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.429894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.430696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.431003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.432655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.441479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 318/649] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.442414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.468538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.503693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f29a9-c301-4dc3-b00d-7dd333e274f8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.538913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.539488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.539770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.540045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.552489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318f27-a218-44ed-b289-235ca82134e0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.590139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.591047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.591513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.591928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.592957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.593407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.595095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.595895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.612417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 319/650] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.612417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.613851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.615392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.694610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.714569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f950ea3-fc17-440f-b81c-ab02ef376b84 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.720790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 332/651] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:34.720790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.753515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.754206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.754484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.754752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.755479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.755826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.757223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.775056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 320/652] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.775056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.812837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:34.824228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d369c-01e3-492f-a465-522966f45d12 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.861501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.862291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.862573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.862848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.873256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.874715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.883065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706990e-f542-4105-9772-3866a39bd18c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:34.896366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:34.897427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:34.898409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:34.898409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:34.898409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:34.898409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:34.900013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 333/653] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:34.900967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.924574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:34.925287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:34.925567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:34.925838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:34.926686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:34.927398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:34.928628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:34.944631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 321/654] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:34.944631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:34.949878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.037952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba199-7224-4706-92f5-940365d6d5c9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.079273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.080435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.080978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.081498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.085617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948f14-473b-4263-a12b-8fc00493632f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.092509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.096537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.098424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:35.119281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:35.120268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:35.121377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:35.121377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:35.121377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:35.121377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:35.121377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 334/655] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:35.122289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.133760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.134884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.136155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.136654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.147004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.149185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:35.166977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:35.167998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:08:35.168978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:35.171795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 322/656] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:35.177186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.253992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.262620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6937f-a813-440b-8f8d-9e8ed7445938 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.296827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162166fc-a0f6-4b30-8e33-ed2dd4e3985f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.303621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.304758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.305053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.305336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.306431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:35.306751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.308457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.321323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 335/657] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:35.322138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.336861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.337901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.338483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.338760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.339476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:35.339780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.341281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dc224-dc7d-4960-aca5-ba2a17c1ef08 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.358053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 323/658] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:35.358053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bb3d-2f30-446c-9d8e-f60bc894d2e3 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.358053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bb3d-2f30-446c-9d8e-f60bc894d2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.358053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bb3d-2f30-446c-9d8e-f60bc894d2e3 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.358053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 324/659] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:35.358487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979bb3d-2f30-446c-9d8e-f60bc894d2e3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.387188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52c34-3f2b-43b6-981f-ce5d4d8d07d3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.470502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979bb3d-2f30-446c-9d8e-f60bc894d2e3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.477644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.478613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.479111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.479591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.480660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:35.481196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.482850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.495317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.507079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 336/660] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:35.507079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.511105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.511887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.512168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.512435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.520534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.522010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-fda329c7-f769-4f51-a073-ff6ed361e4a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:35.538356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:35.539506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 09:08:35.540784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:35.544229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 325/661] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:35.546150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.546437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.546712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.566393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:35.568009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:35.621475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef40f-8d4d-48c3-a7ee-e26d4e179d6e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.633104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.639083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 326/662] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:35.639766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.670723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.671509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.671787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.672844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.679893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.681358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:35.695284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:35.696441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 337/663] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:35.697395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.752599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8e735-d011-4cd3-a80e-b4c4ce577818 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.768761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.790596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.791242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.791523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.791795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.804602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a51ef-a966-48af-aa3f-69fb354236ca None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.838548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:35.839210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:35.839487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:35.839758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:35.840602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:35.840909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.842336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.860606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 338/664] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:35.865049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:35.875889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:35.877464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:35.894646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:35.966470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0960b-c3dc-4da9-bdad-37181914596e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:35.988340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 327/665] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:35.989460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.002854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.003639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.003921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.004189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.008321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.016641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:36.024711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:36.025811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 339/666] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:36.026660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.112473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721c51-19f4-4c15-a34e-0b257111d3dd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.126740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.139463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d676b4d-bf0a-4005-97ef-41d6b6b050b1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.149505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.150205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.150486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.150858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.151645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:36.151954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.153431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.165371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 328/667] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:36.166397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.182942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.184147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.184936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.185367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.250732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.259825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.261543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.280428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940357-bb38-427e-bac5-884842a7ab8a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.317209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.317995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.318274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.318546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.330428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.331881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:36.348834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:36.349785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:36.350706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:36.350706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:36.350706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.350706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:36.350706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 329/668] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:36.351003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.366581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.383354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 340/669] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:36.383354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.463854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128027-5282-4a96-972a-e69a23ea6574 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.488250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.492560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353fe8-0524-404c-b3b1-beed9f05cac8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.496737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.500888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.513810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 330/670] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:36.513810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.537644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.538510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.538858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.539507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.540415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=7025724d5c31460db223411aab77003a) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:36.540831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.542335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.548729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 341/671] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/role_assignments?user.id=7025724d5c31460db223411aab77003a&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:36.549426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.610146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.655252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7c3e7-69e1-4758-8d63-c49b730bfc41 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.667359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e621e-9e2b-4ca4-9276-25b97d7e493e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.704358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.705596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.706470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.706893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.708070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:36.708576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.711088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.715017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.715685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.716039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.716387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.728234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 331/672] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:36.728507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.729606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.730298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:36.761846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:36.763023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:36.764109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:36.764109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:36.764109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.764109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:36.769613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 342/673] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:36.769613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.808040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.880540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5791be-4162-4b0a-8b2f-bec901003fd9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.901522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a4b60-3949-4871-b461-aa6a90f714e6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:36.916698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:36.918293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.919428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.919711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.919989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.932044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.933521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.942005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:36.943019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:36.943443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:36.943857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:36.951508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:36.952071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:36.953123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:36.954322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:36.954322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:36.954322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:36.954322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:36.954322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 332/674] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:36.954674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:36.956424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:36.971774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:36.972872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:36.973960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:36.976150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 343/675] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:36.977106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.080781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.083947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe99a5-301a-423d-bbbb-497da4307b1d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.120164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.121111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.121535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.121648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00fd35-5651-470e-8e24-1c910cdcd9d4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.122072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.122944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.123388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.124906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.142175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 333/676] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.143224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.163784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.164944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.165303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.165644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.166439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.166912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.168885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.188661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 344/677] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.189737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.217820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.278259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a1678-1135-4b46-8b05-f6e9546ea8b2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.301127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487a23-4df2-48f3-875a-90a94fd7f8c5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.322894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.323837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.324349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.324862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.325866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.326345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.327929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.341419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 334/678] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.342555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.344869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.345709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.345979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.346258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.353352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.354764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.366774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-06e12428-2e5a-49c2-b7ab-c81ba307635f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:37.368482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:37.369383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:37.370259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 345/679] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:37.371284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97709e-ea9f-46ef-aa0e-8827b02b9088 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.371284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97709e-ea9f-46ef-aa0e-8827b02b9088 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.371440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97709e-ea9f-46ef-aa0e-8827b02b9088 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.376731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 346/680] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:37.380757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97709e-ea9f-46ef-aa0e-8827b02b9088 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.457361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9518198-ab06-4dde-8a2f-32d4a1c4781a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.467597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.484383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97709e-ea9f-46ef-aa0e-8827b02b9088 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.496206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.497024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.497303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.497571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.505770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.507229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.520747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.521400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.521683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.521958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:37.522144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:37.523131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:37.529015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:37.529575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=c7d851c1cb8b495a87692b8e8970f851) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.529575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.529702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 335/681] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:37.529702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.529828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498bcf3-da02-44af-8697-a2419c7ea24f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.545467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 347/682] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/role_assignments?user.id=c7d851c1cb8b495a87692b8e8970f851&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.545467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.545467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.545467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.555010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:37.556264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:37.643806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.647629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 348/683] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:37.648607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.689058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe322e1-7827-4ec4-9257-216057e987f6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.737292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.737886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.738167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.738440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.739713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.740024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.740890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.743260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.756796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 336/684] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.757947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.771261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ce2c-87d9-438c-be85-4d79c09e3087 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.813301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.814290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.814728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.815004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.815834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:37.816140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:37.818360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:37.842277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 349/685] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:37.848042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.890347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ed12-2433-4618-a135-7268fe02f906 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:37.928011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:37.928857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:37.929167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:37.930073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:37.937813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:37.941918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:37.949725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:37.952396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 337/686] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:37.953772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:37.965756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81869d-1a60-4669-85e1-5954ff208828 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.003114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.003703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.003987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.004265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.007852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=943602b745284e6b9a53ff217a9ee0b4) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:38.007852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.007852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.011587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:38.022078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 350/687] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/role_assignments?user.id=943602b745284e6b9a53ff217a9ee0b4&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:38.023345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.127744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdcd8a-6cae-42a6-8dc2-e3d58b665363 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.198550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ddee-00d7-4c9a-b899-52c650576a80 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.207333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.208015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.208298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.208574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.236953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.238060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.238542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.239066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.240838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:38.241374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.243043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.253271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 351/688] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:38.259473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.307032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:38.309825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.311506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.342865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:38.398108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb00e82-9b3f-424d-acfe-bd4f69fbf0d1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.430549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 338/689] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:38.436442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.446256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.447391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.447926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.448396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.469674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.471654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:38.494359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:38.495363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:38.496239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:38.496239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:38.496239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:38.496239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:38.506342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 352/690] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:38.513585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.554024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961eeb-ae42-45b8-9e6b-fb07367b7289 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.564965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:38.596627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.597430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.597708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.597980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.622928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.624372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:38.652098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:38.653107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:38.654093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:38.654093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:38.654093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:38.654093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:38.665328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 339/691] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:38.668893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.734170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa47b7e-e85e-4c60-a582-e95f4919d51b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.761130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:38.802937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.803601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.803881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.804152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.804913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:38.805216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.806637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.833983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 353/692] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:38.838928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:38.849487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6ff8-4c01-4556-8a49-d5d8c0706911 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.957307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bbb04-c5dc-4bd6-a14f-ac7e57752ab3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:38.962052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.962738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.963021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.963297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.964031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:38.964335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:38.965728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:38.992762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 340/693] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:38.993984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:38.994830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:38.995106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:38.995428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:38.998172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.018748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.018748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.025857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:39.045507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:39.046705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:39.047787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:39.053432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 354/694] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:39.055328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.122867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.170886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08577e33-5a27-45c9-924b-622a3b3e4d0a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.212994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ed3e-bce7-4c75-bf09-fb4e7ccb6938 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.214510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.215468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.215756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.216030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.224173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.226011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-084b5c6f-4419-48b9-9a3c-03717b0d21c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:39.244283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:39.245226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:39.246100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 341/695] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:39.247133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f290c-e754-4645-9a58-cbe044b7c846 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.247741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f290c-e754-4645-9a58-cbe044b7c846 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.248021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f290c-e754-4645-9a58-cbe044b7c846 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.253655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 342/696] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:39.253655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f290c-e754-4645-9a58-cbe044b7c846 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.259562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.260230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.260950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.261235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.262076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:39.262391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.263827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04992c-b761-4c4f-85c0-2474b578ef0b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.282710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 355/697] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:39.283903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.284178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.284458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.305098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:39.306221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:39.337179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.372240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f290c-e754-4645-9a58-cbe044b7c846 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.376626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 356/698] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:39.376626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.400981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.434700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.435419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.435708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.436044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.436785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:39.437091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.439402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.452641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 343/699] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:39.454064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.482240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d9a0-dcbf-4c08-bdf7-46ae6b65efd5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.521482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.522390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.522819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.523239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.568554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95e73-a6cf-4afb-971a-a0b92ef483b3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.594763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.596462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.601253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.606895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.607810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.608092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.608362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.620255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.621491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.622738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:39.643070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:39.644485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:39.645660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:39.645660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:39.645660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:39.645660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:39.648862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 344/700] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:39.649824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.712423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 357/701] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:39.713730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.769895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88661-fd32-4933-b711-fb3aa0d70e80 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.806566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.807280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.807772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.807931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.808787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=327a3409391e4d558aeab965401f48e0) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:39.809146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.810661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.817581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181f-1745-49e9-9365-a95188f50c6f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.818043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 345/702] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/role_assignments?user.id=327a3409391e4d558aeab965401f48e0&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:39.818880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.843378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.856111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.859035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.859035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.859035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.859035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:39.859035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.861259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:39.874490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 358/703] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:39.879524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:39.880653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:39.938962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a88f-c684-4fff-87f8-46f9a20124e8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.977811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:39.978666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:39.978958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:39.979230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:39.990134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e46968-8dfc-4df8-b91b-ead4d4e57871 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:39.992559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:39.994152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:40.014738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:40.020072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:40.021046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:40.021046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:40.021046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:40.021046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:40.021046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 346/704] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:40.022136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.031532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.032481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.032977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.033448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.034352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.036023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8aa8a-302a-4e64-8543-4d6e6f16210d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.138353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.142507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff4707-cd52-43b9-822a-c541997147ce None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.166538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 359/705] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:08:39 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:40.168167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf1ac-5e38-447b-83a0-f3a08e342578 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.168167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf1ac-5e38-447b-83a0-f3a08e342578 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.168329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf1ac-5e38-447b-83a0-f3a08e342578 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.178093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 360/706] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:40.181005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.181774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.182376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.182666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.183368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.183487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:40.183656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.183762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.183935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.185058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.199007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 347/707] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:40.205155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.216455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:40.217745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:40.258309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.316859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 361/708] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:40.337511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.344211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c15d6-f19f-4aad-8c7b-0d01f0ead0c2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.405804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.407087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.407372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.407648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.416215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.417750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:40.434776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:40.435767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:40.436741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 348/709] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:40.443207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.470649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.479708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab149b-5c22-4121-9a54-0acd4a6e63e6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.521803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.523242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.523615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.524487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.526040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:40.541776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:40.542840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:40.543685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 362/710] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:40.549596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.564346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68edc-8c4b-4f2b-ad38-dba39b1e3e21 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.586998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.602992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.610529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.620691 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 349/711] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:40.626978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.669208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee50988-cc7b-4d15-ae74-9f75036a0b45 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.713073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.713842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.714880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.714880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.715259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:40.715635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.717940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.726961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 363/712] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:40.731300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.732664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.755265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910d768-a775-4f3c-9428-4a5fce3f3644 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.793924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.794771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.795281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.795654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.796678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:40.797263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:40.798881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b201-b1ee-4e65-937e-b7016f5535bd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:40.805517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 350/713] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:40.806870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213dbe0-96c2-480d-927c-636caa2852c8 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.807266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213dbe0-96c2-480d-927c-636caa2852c8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.807640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213dbe0-96c2-480d-927c-636caa2852c8 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.809497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 351/714] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:40.816624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:40.816624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:40.816624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:40.847614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:40.848862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:40.902299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:40.914133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb244f9-9720-4cb8-b47c-7113b509b989 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:40.970248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 352/715] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 09:08:40.992861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:40.998815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:40.999869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:41.000922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:41.001405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:41.007784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:41.008390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:41.016609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:41.025542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 364/716] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:41.088311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:41.111850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bbd42-f07f-4753-95ba-401036802317 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:41.150567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:41.151367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:41.151642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:41.151911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] PATH_INFO: `/v3/services/network` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:41.152566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:41.153963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:41.168563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:41.169734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:41.170642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 353/717] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:40 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:41.176263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:41.326253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e262d-f3b3-4622-9ae8-f1005e499513 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:41.363131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:41.364118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:41.364597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:41.365045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:41.365916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] RBAC: Adding query filter params (name=network) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:41.366393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:41.367946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d526ea-8313-4c30-a449-01f746c8b2ad None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:41.373281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 365/718] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:41.379519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:41.482655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71f59-ecca-4555-8dca-9289fb49a312 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:41.520696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:41.521744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:41.522202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:41.522647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:41.523528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] RBAC: Adding query filter params (type=network) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:41.524008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:41.525606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff23a7f-bbcf-4b76-8662-8f968ecc6141 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:41.532022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 354/719] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:42.848886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea160-f895-4aff-86fd-fac7a17b4caf None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:42.849417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea160-f895-4aff-86fd-fac7a17b4caf None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:42.849533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea160-f895-4aff-86fd-fac7a17b4caf None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:42.851133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 366/720] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:42.856242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:42.856670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:42.856853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:42.862520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:42.881103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:42.882292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:42.980554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 355/721] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:43.022167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea160-f895-4aff-86fd-fac7a17b4caf None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.050159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:43.158329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea160-f895-4aff-86fd-fac7a17b4caf None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.200070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.200895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.201174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.201442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.217643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-cc7f555b-8ffc-47e2-a9d0-7bce3f24072c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:43.225462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:43.226382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:43.227217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 367/722] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:43.232368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.345103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739df51e-3fa1-4a2a-9476-a35928ec4f59 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.348602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a68f-3fa1-4315-869e-236491d6fe66 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.348883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a68f-3fa1-4315-869e-236491d6fe66 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.349154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a68f-3fa1-4315-869e-236491d6fe66 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.350566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 368/723] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:43.358065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.358351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.358628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.386762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:43.388118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:43.408095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:43.417545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.418238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.418521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.418794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.422696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:43.422696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:43.422696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:43.432408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 356/724] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:43.438806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.526993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 369/725] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:43.561748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.574736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e6b2f-cda7-482f-b82f-9e05c5d9919c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.585645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:43.669997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.670656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe9bc6-6305-419e-bad8-3e0997052479 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.670862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.671068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.671324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.693693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:43.695158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:43.708460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.709731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.710013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.710282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.721685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:43.733038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:43.733926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:43.735368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:43.735368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:43.735368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:43.735368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:43.735368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 357/726] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:43.737188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:43.738098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:43.738961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 370/727] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:43.741543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.756340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.824979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:43.882041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:43.900623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6e78f-652d-4208-8849-cd0101222aa9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.911299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fdfe6-4f57-4429-b9b7-af31d9bc1f02 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:43.940515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.941226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.941489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.941957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.942690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:43.942995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:43.944416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50835dc6-7497-4ca6-9144-c262611f2e81 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:43.950954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:43.951683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.951900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.952184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.952967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:43.953308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:43.954692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d3b3-8490-484c-b6e9-10556effd8d9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:43.958654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 358/728] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:43.960554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578941d-c870-4951-b5a0-58f26a608a97 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.961306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578941d-c870-4951-b5a0-58f26a608a97 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.961750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578941d-c870-4951-b5a0-58f26a608a97 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.966989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 371/729] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:43.966989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d91084-8558-4c9f-890c-3db4cbac00c0 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:43.966989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d91084-8558-4c9f-890c-3db4cbac00c0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:43.966989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d91084-8558-4c9f-890c-3db4cbac00c0 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:43.967291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 359/730] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:43.967291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578941d-c870-4951-b5a0-58f26a608a97 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:43.970771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 372/731] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:43.971882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d91084-8558-4c9f-890c-3db4cbac00c0 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.105690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578941d-c870-4951-b5a0-58f26a608a97 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.105930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d91084-8558-4c9f-890c-3db4cbac00c0 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.114399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.142855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.143240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.143651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.143934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.144031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.144208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.144304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.144556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.146998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.153876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.155269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.155380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.156732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-15d14140-0af3-4245-9a88-cdaaa2fd8932 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:44.169946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:44.170921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:44.171941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 360/732] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:44.173122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.173122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.173416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2e0c11d6-4cb5-42e3-b42b-ee51ed056665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:44.176751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:44.177604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:44.178473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:44.178473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:44.178473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:44.178473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:44.180163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 373/733] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:44.182751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.183192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.183594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.194528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:44.195652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:44.201333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:44.202493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:44.268993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 361/734] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:44.269824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.284237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 374/735] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:44.284237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.363495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.369070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94aa5-24ce-4de7-9055-a59f30e2558a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.380116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b21df-70a0-47e4-9185-5a6c96f10290 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.397042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.403551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.404223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.404510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.404817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.405663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:44.406042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.407488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a677b76-4ccb-49e0-80b4-d49806ae81e3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.413442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.414096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.414379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.414654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.415373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:44.415676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.417091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.425465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 362/736] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:44.426720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4f95-127c-4928-a633-f989f815b07b None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.427006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4f95-127c-4928-a633-f989f815b07b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.427280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4f95-127c-4928-a633-f989f815b07b None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.428513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 363/737] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:44.430496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c4f95-127c-4928-a633-f989f815b07b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.434391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 375/738] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:44.435350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.532375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c4f95-127c-4928-a633-f989f815b07b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.534352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28107bc0-55c5-4bcf-97e4-84e75d35c36c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.566196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.566847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.567130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.567404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.567857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.568113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.568662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.568951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.569230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.569515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea53758-46ed-41ef-ae00-819d16eef55d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.579306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:44.586982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:44.589649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 376/739] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:44.590692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.628420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.686414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63ab5-5c16-4dbc-8bec-14cbdec38684 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.696546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 364/740] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:08:44 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:44.697915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.698211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.698485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.716446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:44.717879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:44.730221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.733949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.733949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.733949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.733949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.804717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 365/741] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 09:08:44.804717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.813755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.815464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.852626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:44.923654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509c1b-bee2-4c6b-ab98-f4f1884ceec4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:44.930088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 377/742] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 09:08:44.930958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.959855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:44.960692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:44.960973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:44.961249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:44.969598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:44.971053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:44.986663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:44.987633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:44.988617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:44.989658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 366/743] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:44.989658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:44.996128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.039408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0adda2-3b56-4210-8e10-e660a320a19b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.073494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.074088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.074371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.074648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.075389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:45.075696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.077117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.078610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.090813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 378/744] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:45.090813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.110103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41365e-7580-47e9-8abc-f5acb1776033 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.153070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.153887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.154170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.154443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.155679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.157588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.187276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3dab0-4a04-412f-8cc0-fa5793804b67 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.220106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.220925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.221207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.221482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.233286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.235310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.235813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 367/745] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:08:44 2022] POST /identity/v3/services => generated 240 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:45.236742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:45.255560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:45.256560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:45.257499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:45.257499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:45.257499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:45.257499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:45.257499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 379/746] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:45.258771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.320286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.327116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.386215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd8e47-c5eb-4c75-942f-95d27f8f7504 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.394404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9998c-70f2-4302-b61b-e05cc982d1a0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.427336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.428242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.428516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.428911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.442603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.443824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.445380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.446353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.447040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.447313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.448232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:45.448545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.449998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:45.463762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:45.464763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:45.465668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:45.465668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:45.465668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:45.465668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:45.465668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 380/747] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:45.466467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.474681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 368/748] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:45.475555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.583073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.586356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb822948-2eeb-4b8d-b67d-553d7bca5e73 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.610233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882dcb-21d1-47c8-a663-82e0975c038e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.629343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.633849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.634995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.635909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.636744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.638009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:45.638685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.640973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.651317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.652831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.653548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.653977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.654946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:45.655424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.657567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.660796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 381/749] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:45.662196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.678744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 369/750] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:45.678744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.789494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463144-99c7-4707-9fd2-e2a86ede37ab None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.805108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af254c-bd7b-4c5b-b91d-3a70be588647 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:45.825502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.826135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.826516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.826926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.834892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.836375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.850734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:45.851663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:45.852097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:45.852241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:45.853296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:45.854401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:45.855291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:45.860962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 370/751] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:45.862008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:45.894241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.908602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:45.937877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:45.940009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:45.983572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2a9f1-3b3d-488f-bf92-a37096261844 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.038004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.039474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.039966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.040431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.051842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.053792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.059714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 382/752] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:08:46.060749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:46.070862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:46.071788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:46.073076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 371/753] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:46.077450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.138301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.181633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.185181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d787b-e850-401a-ab16-68d47d871bfe None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.185299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545f1e7-a280-42ff-b9e3-0d2d40f34522 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.221739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.222621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.222979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.223352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.224016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.224958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.225526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.225998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.227044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:46.227528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.229183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.234400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.235927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.242807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 383/754] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:46.243879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:46.254906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:46.255833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:46.257023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:46.257023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:46.257023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:46.257023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:46.261036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 372/755] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:46.261954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.356608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66bdb-d2cc-4aef-9d68-3b076501988c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.365155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd7ad9-a6e6-4496-b2a0-ef7caafe4511 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.379329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.400733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.400733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.400733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.400733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.401115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:46.401115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.401115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.402145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.402866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.403225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.403572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.404630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=327a3409391e4d558aeab965401f48e0) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:46.404935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.406968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.413422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 373/756] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/role_assignments?user.id=327a3409391e4d558aeab965401f48e0&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:46.414384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.423847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 384/757] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:46.432842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.438491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.537279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4f6d-11de-4a41-98ee-5a7ada81ffd4 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.587868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.589229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.589774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867624e0-45df-4743-86e1-49eaf3c90a78 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.591698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.592089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.592965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:46.593438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.601223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.630141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 374/758] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:46.636520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.683599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.684505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.684816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.685094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.709549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.711185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:46.731645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:46.733717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:46.736406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:46.736406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:46.736406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:46.736406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:46.738974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 385/759] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:46.741257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.741395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.774718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:46.777873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67e2d-a4d0-44c7-98cd-b8f8cfa682db None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.839046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.840232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.840728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.841184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.851463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.853186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.870426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884673-0f94-4dcc-bf33-1e016cb9658b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:46.884731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:46.885912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:46.886805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:46.889233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 375/760] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:46.905426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:46.911288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:46.911964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:46.912247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:46.912519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:46.914012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:46.914012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:46.915059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:46.936775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 386/761] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:46.936775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.040098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:47.052021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d790da-2e9a-431a-a796-1e73c00e5a1f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.072933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bdb3-bb1c-462f-a846-71d61c3a8bc3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.098124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.099538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.099889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.100241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.111300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.112859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.117523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.118449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.118936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.119671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.120776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:47.121293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.122911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10648f8-7deb-41e8-8e61-0da97758479d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.125482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-afd1f84c-ee02-4f3f-b319-3ef97ddccb66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:47.128192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:47.129278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:47.130462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 387/762] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:47.131139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088cad6-7753-4abe-b66a-e3ae24e187d1 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.131301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088cad6-7753-4abe-b66a-e3ae24e187d1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.131540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088cad6-7753-4abe-b66a-e3ae24e187d1 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.137754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 388/763] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:47.138838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088cad6-7753-4abe-b66a-e3ae24e187d1 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.149024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 376/764] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:47.150057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.150331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.150609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.173710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:47.175759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:47.255745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088cad6-7753-4abe-b66a-e3ae24e187d1 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.274783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 377/765] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:47.276140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.294663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51901649-9595-4e37-ac80-3584b959fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.295382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901649-9595-4e37-ac80-3584b959fada None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.295666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901649-9595-4e37-ac80-3584b959fada None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.295948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901649-9595-4e37-ac80-3584b959fada None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.296929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51901649-9595-4e37-ac80-3584b959fada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:47.297246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51901649-9595-4e37-ac80-3584b959fada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.298882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51901649-9595-4e37-ac80-3584b959fada None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.313447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 389/766] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:47.314343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51901649-9595-4e37-ac80-3584b959fada None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.332340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:47.392487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1853e6-5c06-4fbe-a952-4416f115bfe5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.416872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:47.426618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51901649-9595-4e37-ac80-3584b959fada None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.428748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.429410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.429690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.430698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.430811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=37821819ba394afda7a92b7aa99096b2) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:47.431092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.432516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.444724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 378/767] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/role_assignments?user.id=37821819ba394afda7a92b7aa99096b2&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:47.445756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.476884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.478141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.479018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.480089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.481116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.482772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:47.496523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:47.497573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:47.498420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:47.501520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 390/768] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:47.507071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.649072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:47.663513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9e25-db55-42bd-be6c-b77ca70d84c1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.666678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34522c-158f-4ec5-ae38-c8d94934b674 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.708742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.709572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.709880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.709983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.710200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.710855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:47.710969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.711164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.711260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.711611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.712543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd65bf-fb99-4551-95e7-d1fadf50a666 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.713863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=dff6b630f6be464aa6cdcc73da587e4f, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.715298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530157-7f98-4b4f-ac2d-b8ce9925f39b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.717549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 391/769] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:47.719553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311d45-6fe5-4864-b9ba-382ab7e6db84 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.719843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311d45-6fe5-4864-b9ba-382ab7e6db84 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.720649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311d45-6fe5-4864-b9ba-382ab7e6db84 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.722188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 392/770] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:47.724710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311d45-6fe5-4864-b9ba-382ab7e6db84 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.792812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 379/771] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:47 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:47.794259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.794555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.794836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.845754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:47.847051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:47.851048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311d45-6fe5-4864-b9ba-382ab7e6db84 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:47.895233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:47.896616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:47.897129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:47.897660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:47.898661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:47.899228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:47.900925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:47.908799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 393/772] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:47.916409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:47.919953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.028987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.044795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cbd-288f-4e33-bf7a-d481c6c3ff1a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.073264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 380/773] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:48.096449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.097171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.098049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.098049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.098495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] RBAC: Adding query filter params (interface=public, service_id=a9f4ffadc1654edd80357119acb429a0, region_id=RegionOne) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:48.098802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.100173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.106125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:48.114310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 394/774] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:08:47 2022] GET /identity/v3/endpoints?service_id=a9f4ffadc1654edd80357119acb429a0&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:48.118642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:48.236714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.242756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987edb42-7517-461c-b39e-04ca07b09804 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.284540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.286130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.286625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.287154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.288320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.290021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:48.304075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:48.305112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:48.305944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:48.312820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 381/775] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:48.315697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:48.332131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133360fc-e55f-4a39-9a00-41506a128ff7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.406856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.409155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.410923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.411420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.412312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.415466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.416012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.431533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 395/776] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/services => generated 772 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:48.494300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e2f8-dcf1-4211-88b8-a22eaf5cffe0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.542871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.543939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.544451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.544960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.545948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:48.546470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.548150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.558710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 382/777] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:48.560441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:48.676295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc45f-61f1-41f4-993f-64462792e40d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.707556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.714239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.715161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.715647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.716123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.717084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:48.717593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.719756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.725423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 396/778] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:48.731817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:48.848352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed05a63-1777-4a56-92b9-b9f3656368f0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:48.856160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:48.884999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:48.886291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:48.886779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:48.887275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:48.888413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] RBAC: Adding query filter params (interface=public, service_id=6a5cdc2776c6409db1928a7d2cf91128, region_id=RegionOne) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:48.888992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:48.891268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5012a-2603-4117-90d9-c2ab260fb795 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:48.904467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 383/779] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/endpoints?service_id=6a5cdc2776c6409db1928a7d2cf91128&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:48.907357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:49.025531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce576-432e-483c-9e11-189fe07588e7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:49.062214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:49.063824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.064340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.064836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.065726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:49.067482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb537e9c-1227-40d8-8dbb-33ab8c565815 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:49.077469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 397/780] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:08:48 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:49.221538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754d7-0e65-4554-b3e6-eae89571f5be None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.222535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754d7-0e65-4554-b3e6-eae89571f5be None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.223015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754d7-0e65-4554-b3e6-eae89571f5be None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.224869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 384/781] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:49.230125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.230697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.231230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.253339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:49.254874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:49.316557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:49.332024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 398/782] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:49.360168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5754d7-0e65-4554-b3e6-eae89571f5be None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:49.471501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5754d7-0e65-4554-b3e6-eae89571f5be None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:49.517386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:49.518613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.519149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.519637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.531660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:49.539698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:49.540640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:49.541483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:49.548522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 385/783] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:49.549844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:49.667287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe3aa7-05f8-4b01-ac34-e82eeb1013f2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:49.703545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:49.704606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.705101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.705596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.706589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:49.707114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:49.708818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:49.717917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 399/784] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:49.725672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:49.773038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:49.862683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebceb6-5541-427a-b83d-ea5e0a9d55a8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:49.909377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:49.910228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:49.910514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:49.910784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:49.922158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:49.923592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:49.946464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:49.947418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:49.948286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:49.948286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:49.948286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:49.948286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:49.948286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 386/785] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:49.953798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.020868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:50.086731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8388a-4ebe-401e-8d0e-6334f5e95e48 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.134142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:50.134954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.135465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.135749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.136493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:50.136825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:50.138217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af76b4b-dc05-4697-987a-d71c31830f37 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:50.160084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 400/786] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:50.165138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.210900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe260-fa28-410e-a23e-e7f421d905a4 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.211242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe260-fa28-410e-a23e-e7f421d905a4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.211461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe260-fa28-410e-a23e-e7f421d905a4 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.212980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 401/787] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:50.218020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.218300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.219112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.241404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:50.242633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:50.315717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:50.345540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52f47-16a6-4f5b-a47a-bf083e0f047c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.367552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 402/788] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:08:50.393681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.402672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:50.403491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.403775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.404051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.417996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:50.419456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:50.439434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:50.440415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:50.441396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 387/789] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:50.446461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.451384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:50.542020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541acf1-b34e-491e-9c71-39b75e38e16f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.611685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:50.612454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.612771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.613040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.636817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4d07e-28b7-4ca4-a631-c6712919c28f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.637348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-cf91cf5b-c974-44ca-acce-1325c06322e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:50.645094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:50.646043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:50.646903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 403/790] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:50.648081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734a0c-e28a-4594-ad14-3ff279c95c90 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.648081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734a0c-e28a-4594-ad14-3ff279c95c90 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.648324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734a0c-e28a-4594-ad14-3ff279c95c90 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.652603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 404/791] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:50.657425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68734a0c-e28a-4594-ad14-3ff279c95c90 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.674537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:50.675372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.675657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.675934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.676854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:50.677252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:50.678792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2e18-f6c7-4ad7-be7e-0862a669edc4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:50.694599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:50.699912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 388/792] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:50.701455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.702025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.702479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.725751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:50.727239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:50.766509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:50.770755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68734a0c-e28a-4594-ad14-3ff279c95c90 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.808650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:50.809518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:50.809705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:50.809972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:50.810715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:50.811055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:50.813040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:50.822109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 405/793] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:50.822925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.823775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 389/794] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:50.832573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:50.956490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6d703-4faf-40a6-a066-97b98f3ec877 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:50.974889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed258ac1-0fa5-4e76-88b2-6e939a9d18a4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.003964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.004825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.009260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.009782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.011373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.019830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.020685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.020968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.022247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.030054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.033885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.034546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.036950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:51.057812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.058773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.059799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:51.059799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:51.059799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.059799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:51.060031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.061181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.062604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:51.062604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:51.062604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.062604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:51.062604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 390/795] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:51.063164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.065304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 406/796] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:51.066709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.086106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.178486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf18fe-eb90-4d38-be15-93c9c9a5ee76 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.200805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fca60-ed12-48b5-aece-c609ea9f3596 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.219998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.221700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.222169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.222642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.237626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:51.239443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.240169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.240517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.240887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.241694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:51.242066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.243508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:51.247894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.248827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:51.249718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:51.253123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 391/797] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:51.254933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.261718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 407/798] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:51.262641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.313240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.354592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.377903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b61ef-58ca-4f38-a82e-8fc822d41e66 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.383266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a7fc1-a54e-4b96-a3dc-ef2110e2e8df None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.415860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.416915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.417407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.417900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.418827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:51.419352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.420167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.420853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.421134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.421409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.421918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25293e-115a-40a3-851c-6db2e1c0041f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.422170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:51.422473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.423840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.432290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 408/799] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:51.433025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.435697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 392/800] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:51.437552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bb67e-ec49-4426-9471-b39f5fc1ef12 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.437843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bb67e-ec49-4426-9471-b39f5fc1ef12 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.438124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bb67e-ec49-4426-9471-b39f5fc1ef12 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.439507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 393/801] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:51.444205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bb67e-ec49-4426-9471-b39f5fc1ef12 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.553889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bb67e-ec49-4426-9471-b39f5fc1ef12 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.555819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c234-15b9-4f7b-bd45-254baa57bfb1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.555963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.591788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.592416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.592654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.592899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.593179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.593273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.593551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.593829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.596929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.603000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.604513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.604750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.611167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:51.620496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.621554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 409/802] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:51.628220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-26da00fd-3a36-4f3f-bd92-87286991036b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:51.628914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.629730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.630628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:51.630628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:51.630628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:51.630628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:51.632570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 394/803] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:51.634146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.634447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.634733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.653572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:51.654769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:51.732627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 395/804] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:51.732627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.748368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685c6d7-1c66-4eb6-8c55-71ef9314d4bb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.785821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.786617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.786901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.787224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.794056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.795509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:51.810379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:51.811486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 410/805] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:51.812939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.834892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.850959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123edfbf-eb78-4a41-841c-39aa77369ef4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.878027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:51.888867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.889542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.889826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.890102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.890968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:51.891280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.892742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.910404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 396/806] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:51.911231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:51.928326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1604f0-1c17-4b62-99f3-3610d0e46f4d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:51.965551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:51.966215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:51.966499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:51.966767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:51.967481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:51.967782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:51.969747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:51.980909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 411/807] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:51.981694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.014501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325a0c-518d-4672-9237-4bb74f4f90d6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.049709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.050501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.050784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.051063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.051713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.053108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:52.065273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:52.066391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:52.072465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 397/808] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:52.073625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.080673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.081878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6dc8-f56d-4c98-9759-0aad5595dd37 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.098001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.115118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.115766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.116049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.116327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.117151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:52.117459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.118888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fcc71-35c1-4c99-894d-1f1c1f289a06 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.135993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 412/809] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:52.137196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee730709-58e0-4ccd-9f81-9303d770d733 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.137493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee730709-58e0-4ccd-9f81-9303d770d733 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.137768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee730709-58e0-4ccd-9f81-9303d770d733 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.139016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 413/810] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:52.139698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee730709-58e0-4ccd-9f81-9303d770d733 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.173856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667339-2c24-4f58-81f0-c0b5d081c8bb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.207443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.208225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.208509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.208816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.219148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.220597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:52.237124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:52.238386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 398/811] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:52.239591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.239948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee730709-58e0-4ccd-9f81-9303d770d733 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.276940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.277715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.277996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.278273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.285060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.286493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.300754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:52.301329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:52.302284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 414/812] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:52.303230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.339466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.346532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41867a6-909e-4a2f-b521-ff8bce95c9b1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.379722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.380361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.380807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.381311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.381791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:52.382181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.383461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef41a5-acb7-4250-a4e2-dc90e811365d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.388505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 399/813] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:52.389716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.390007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.390880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.404566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74422d84-4bb4-4b82-a5a2-271d5139a09f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.406659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:52.407835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:52.438071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.438945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.439222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.439492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/327a3409391e4d558aeab965401f48e0/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.441505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=327a3409391e4d558aeab965401f48e0, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.442946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.472745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 400/814] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:52.473566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.501763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 415/815] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:52 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/327a3409391e4d558aeab965401f48e0/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:52.502583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.528473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.584662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c8e62-e1e1-4705-8a0a-3406bb9058bd None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.616036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.622396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c400b4-a4b8-42d0-bf7e-a0fbd523b9db None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.625871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.626584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.626871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.627204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.627949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:52.628260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.629681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.643244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 401/816] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:52.646391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.664516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.665248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.665535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.665821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.666757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:52.667356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.668984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.689476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 416/817] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:52.690625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.765638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d25e4d-7737-4101-9bc8-67932bbd9698 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.781350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.802316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96f66-e339-4f52-9022-30b204828c36 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.808224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.809057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.809344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.809628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.810337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:52.810645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.812291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.821649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 402/818] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:52.822374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.840295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.841768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.842280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.842987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.844288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.845903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:52.859965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:52.861055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:52.861978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 417/819] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:52.862823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:52.892414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:52.937087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d1b6d-65a1-4866-9d86-6fe0cb719a5a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.972491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:52.973435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:52.973719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:52.974121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:52.979499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a83d4-e78e-45bf-b4de-597a31c23af5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:52.981305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:52.982867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:53.000371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:53.001712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 403/820] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:53.002618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.024659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.025321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.028506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.028506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.045636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.101867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.103327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.117953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaef54f-6dc9-49c7-8506-141d4c5a3bc3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.121255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.155443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.157048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.157531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.157995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.159038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.160735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.205856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 418/821] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:08:53.206697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.228966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 404/822] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:08:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:53.234085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.355327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.363232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7418b-a129-49ad-8d55-ba22842c04df None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.372409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1fb9c-ce88-4307-bad4-7526dd2101a8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.389281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.413116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.413813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.413973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.414361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.414777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.415191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.415625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.416057 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.416183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:53.416614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.417720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:53.418183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.418975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.419984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.432634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 419/823] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:53.432634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.439828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 405/824] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:53.440543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.552554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97c96-3b6d-4777-abfd-f8f7ac21c2e0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.568794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b475e-c46e-4c68-9557-d2efa9688ffe None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.597444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.598584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.599087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.599680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.612631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.614300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.614954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.615069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.615257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.615537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.616274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] RBAC: Adding query filter params (type=network) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:53.616576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.618405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.628693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 406/825] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:08:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:53.629973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:53.643228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:53.644213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:53.645153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:53.645153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:53.645153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:53.645153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:53.648663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 420/826] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:53.649718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.653168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.704430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:53.754231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128fa0-9646-478b-8cdd-de8d7b3fa2ee None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.783207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c150-7efb-45cc-80b2-2bac4984cd46 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:53.805208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.806945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.807486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.807914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/37821819ba394afda7a92b7aa99096b2/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.821631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:53.822330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:53.822614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:53.822932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:53.823676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.825136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.828046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=37821819ba394afda7a92b7aa99096b2, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:53.830458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:53.902256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 407/827] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:08:53 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/37821819ba394afda7a92b7aa99096b2/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:08:53.903623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:53.929288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 421/828] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:08:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:54.023066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:54.079739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54726-2f3d-4dc8-a798-41e67771d175 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:54.142246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:54.142938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.143220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.143497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.144245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:54.144553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:54.146003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:54.165135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 408/829] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:54.170948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:54.283312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:54.296148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d705f8-7120-4f3e-beeb-48d37a85f7f8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:54.336224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:54.337324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.337757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.338179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.345886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:54.347539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:54.363277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:54.364922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:54.366047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:54.368183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 422/830] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:54.373852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:54.494867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cae61-d9ac-4492-9000-dd14b885508f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:54.514128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:54.532201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:54.533110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b82ce7-489d-4b0a-bc93-1b88c307268e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:54.554171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 409/831] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:54.556032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:54.667904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952da2-8693-4791-a7c7-66e48148c1ac None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:54.705127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:54.706202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.706715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.707195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.708278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:54.708818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:54.710460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0967637-6ee2-4458-892c-76a6716713ef None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:54.720679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 423/832] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:54 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:54.928169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9a48-d1eb-4fbe-913b-7a9467ca3cd2 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.928535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9a48-d1eb-4fbe-913b-7a9467ca3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.928765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9a48-d1eb-4fbe-913b-7a9467ca3cd2 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.930310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 410/833] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:54.935543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:54.935835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:54.936113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:54.953484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:54.954667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:54.964216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:55.035729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 424/834] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:55.059586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9a48-d1eb-4fbe-913b-7a9467ca3cd2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.166665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9a48-d1eb-4fbe-913b-7a9467ca3cd2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.206788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.207627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.207971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.208248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.220202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:55.228343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:55.229401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:55.230313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 411/835] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:55.236008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.344430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd2f76-f4fc-4bad-b0d1-1b60993ce941 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.379027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.379702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.379967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.380244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.381027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:55.381335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:55.382778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:55.395977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 425/836] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:55.401206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.444469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:55.506715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376df88-e097-4bd2-a5a8-d8a3bcd20b93 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.541846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.542638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.542921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.543197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.554126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:55.555573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-80d0c96c-cd3d-4e7e-bcbb-866d58ab499c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:55.573149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:55.574075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:55.574971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:55.574971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:55.574971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:55.574971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:55.574971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 412/837] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:55.580293 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.602580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:55.632249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c3a-5427-458e-943d-ef4397682650 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.632619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c3a-5427-458e-943d-ef4397682650 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.632860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c3a-5427-458e-943d-ef4397682650 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.634540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 413/838] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:08:55.639374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027607-0960-44b2-a287-4e9db5110220 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.639658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027607-0960-44b2-a287-4e9db5110220 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.639931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027607-0960-44b2-a287-4e9db5110220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.657585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-14027607-0960-44b2-a287-4e9db5110220 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:55.658764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-14027607-0960-44b2-a287-4e9db5110220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:55.688039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a77e-ccb8-49a2-908e-53382e0cedb5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.722526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.723202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.723490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.723769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.724634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:55.724951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:55.726343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:55.743439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 426/839] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:55.744517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.748666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 414/840] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:08:55.769681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14027607-0960-44b2-a287-4e9db5110220 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.792368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:55.831318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:55.878657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a84e7-e812-4f19-99ac-6bdfc38b4b5b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.883078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14027607-0960-44b2-a287-4e9db5110220 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:55.922321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.923113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.923390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.923648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:55.923821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.924297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:55.924654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:55.925006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:55.925344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:55.925696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:55.933129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:55.934686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:55.940104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:55.942086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:55.943116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 415/841] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:55.947668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:55.955666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:55.957174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 09:08:55.959167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:55.964632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 427/842] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:55.964632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.069088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.086839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2bf59-497c-4574-8c70-eb490e85dd9d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.096010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd4d3-f29e-4dca-91c0-e5b8997a4893 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.128438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.129153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.129439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.132681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.132823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.132906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:56.132906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.132906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.137642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 416/843] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:56.143124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.143700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.144566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.144987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.145363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.147713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:56.148264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.150184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.173858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 428/844] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Jun 7 09:08:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:56.179639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.258434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e367ab-c093-438e-9976-95d8c0d70247 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.294425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.295095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.295367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.295652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.296389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:56.296728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.298091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be761-755b-4d15-bdf0-84012d305ff1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.308619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 417/845] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:56.315431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618558d-1d97-4769-bbb4-c3b785ffd778 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.354373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.354950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.355235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.355509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.420918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.432608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944013-a760-43b2-81c7-b148f990fd8f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.432608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944013-a760-43b2-81c7-b148f990fd8f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.432608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944013-a760-43b2-81c7-b148f990fd8f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.432608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 418/846] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:56.440394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.440752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.440984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.445385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.447000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.459739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:56.461081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:56.470096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.543629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 419/847] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:08:56.564899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 429/848] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:08:56.569450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.571847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.679851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.683459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a70ef-51c1-478f-a164-7d59e653a6ed None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.705781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77cc9c-36ce-4d2a-8fa1-1f0e8e6a2ef0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.726970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.727777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.728059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.728331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.741499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:08:56.748848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.750214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.750499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.750778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:08:56.751160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:56.752063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:56.752966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 420/849] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:56.758429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.762137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:56.770870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.772834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:56.797839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:56.799928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:56.802003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:56.802003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:56.802003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:56.802003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:56.802003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 430/850] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:56.805601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.895686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632399c-99a5-4555-bbc7-d9409390f4c2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.941934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.942529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.942814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.943090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.943608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80500ba-fd7d-481c-9a03-58f1fc69313c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:56.943848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:56.944218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.945955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.958461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 421/851] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:56.964319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:56.985386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:56.986144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:56.986498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:56.986837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:56.987641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:56.988175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:56.989674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:56.992690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.004972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 431/852] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:57.016180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.060187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.110790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa2ae0-9bb6-4253-8299-16b0ab5b59ef None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.147816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.148636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.148926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.149204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.151358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847982-2ead-4c97-8d77-1af54b682a3d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.160933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.162397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-df1d2475-2501-42a2-8579-1c9852378293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:57.181726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:57.182683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:57.183578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:57.183578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:57.183578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:57.183578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:57.183578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 422/853] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:57.184360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf5e4-fde4-47f8-bbaf-a6fba71f8159 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.184669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf5e4-fde4-47f8-bbaf-a6fba71f8159 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.184953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf5e4-fde4-47f8-bbaf-a6fba71f8159 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.186646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 423/854] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:57.188360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cf5e4-fde4-47f8-bbaf-a6fba71f8159 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.195826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.196561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.196873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.197150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.205349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.206815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-78e22f06-fe9b-4268-9b74-c918c073e289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:57.221453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:57.222424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:57.223320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 432/855] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:57.224550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.224550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.224550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.241960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:57.243085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:57.299297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.301021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cf5e4-fde4-47f8-bbaf-a6fba71f8159 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.317200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.318145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 433/856] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:08:57.318861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.347595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.348281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.348564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.348873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.349611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.349907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.351298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.364527 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 424/857] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:08:57.365348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.426846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ce8f-fc44-4346-b0c6-a6a66e7714c8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.463379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.463962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.464245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.464524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.465332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.465640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.467091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.479417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 434/858] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:57.480146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2719258-059d-4efe-8929-59a65317eceb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.480813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.528428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.529368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.529713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.530228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.531959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.533020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.533943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:57.546919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:57.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:08:57.549226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:57.553075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 425/859] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:57.554136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.593778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8dde-0a5a-4e71-bef2-e8d1bb2cb868 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.599827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.629713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.630809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.631296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.631759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.640178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.642476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:08:57.657755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:57.658727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:08:57.659667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:57.660297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbea82-fb10-4a43-97b5-84041828195b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.662105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 435/860] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:57.663106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.700212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.700889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.701176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.701452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.702283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=327a3409391e4d558aeab965401f48e0) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.702654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.704079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.710752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 426/861] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/role_assignments?user.id=327a3409391e4d558aeab965401f48e0&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:57.711605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.779528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94ec99-43c8-4d9c-a4fd-f31c44de141d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.787493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.817158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.817841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.818153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.818402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.819137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.819450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.821022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.832753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 436/862] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:57.834389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.851037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351f65c-3d12-4076-b478-766ac2fe40c5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.854107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:57.893665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.894640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.895173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.895622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.896687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.897210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.899056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:57.918678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 427/863] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:57.924735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:57.950032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8bbc-d50d-44ef-8c63-04501373349d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:57.991762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:57.992648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:57.992782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:57.992993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:57.993705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:57.994010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:57.995386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.008672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 437/864] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:58.065781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb7fdc-5b0a-4036-9fdd-41874c3286c9 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:58.106502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:58.107487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.107917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.108348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.145142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:58.189966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:58.191712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.303330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 428/865] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:08:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:08:58.309166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:58.357532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:58.418258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87aabc-4c57-4526-96e0-8e3a8958b817 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:58.452887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:58.453694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.453954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.454229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.465188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:58.466659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-49e86846-6754-421c-8b1f-af348ed90e1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:08:58.484742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:58.485991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:58.487749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:08:58.487749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:08:58.487749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:08:58.487749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:08:58.487749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 438/866] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:08:58.491444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:58.558050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:58.600973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4c3d-33fb-40e3-b20c-019254ab0a64 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:58.638170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:58.638749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.639028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.639298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.640644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:58.640954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:58.642312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.654874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 429/867] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:58.660605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:58.660833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bef56-0478-4ab5-b307-2c2529071d08 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.661123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bef56-0478-4ab5-b307-2c2529071d08 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.661402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bef56-0478-4ab5-b307-2c2529071d08 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.663015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 439/868] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:08:58.667877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.668172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.668454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.685533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:08:58.686703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:08:58.767188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0486-2a8e-4998-9595-57a7ea4bc6a2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:58.776675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 440/869] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:08:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:08:58.797552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:58.799906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:58.811301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:58.812214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.812503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.815051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.817584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:58.820059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:58.821614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:08:58.836508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:08:58.837492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:08:58.839157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 430/870] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:08:58.844234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:58.919968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9f24e-2e2f-4878-a80b-06f9eaae6456 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:58.956785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:58.957477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:58.957758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:58.958031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:58.959222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:58.960203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9376a7-a2e0-4b8c-9673-83957a9ce5c5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:58.980375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4236918-3979-45f0-8143-3c4ac207f7b1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:59.025577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:59.028448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:59.028448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:59.028448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:59.028448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:59.028448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:59.029386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:59.042382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 431/871] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:08:59.048016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:08:59.059323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 441/872] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:08:58 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:08:59.104523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:59.171427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b149e-03e0-4d5b-8de3-61984bf6c722 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:08:59.217897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:08:59.219622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:08:59.220133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:08:59.220671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:08:59.221980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=37821819ba394afda7a92b7aa99096b2) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:08:59.222553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:08:59.224353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156e42-0501-4ff0-9b0b-ecd67609f661 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:08:59.232648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 432/873] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:08:59 2022] GET /identity/v3/role_assignments?user.id=37821819ba394afda7a92b7aa99096b2&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:00.118473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450076f3-b7df-46d8-b186-3cdf902f7e42 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.118990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450076f3-b7df-46d8-b186-3cdf902f7e42 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.118990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450076f3-b7df-46d8-b186-3cdf902f7e42 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.120635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 442/874] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:00.126095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.126867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.126867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.144657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:00.145835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:00.216237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 433/875] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:00.238746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450076f3-b7df-46d8-b186-3cdf902f7e42 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:00.321531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:00.344379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450076f3-b7df-46d8-b186-3cdf902f7e42 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:00.379418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:00.380218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.380499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.380803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.381457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:00.382842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-6fb72359-c54a-47c3-81cd-1e0cbb16fdf4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:00.395710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:00.396765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:00.398137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 443/876] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:00.403876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:00.437651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:00.450250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2d19-cede-4273-920d-9e98a179fdb0 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.450697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2d19-cede-4273-920d-9e98a179fdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.451051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2d19-cede-4273-920d-9e98a179fdb0 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.453291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 444/877] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:00.458526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.458814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.459093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.477371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:00.478572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:00.533066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478964a-91a0-47a1-bd42-47414e3b5d79 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:00.561576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 445/878] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:00.573492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:00.574093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.574371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.574647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.575404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:00.575712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:00.577107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:00.582295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 434/879] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:00.587893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:00.598273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:00.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:00.706086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:00.712755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d9fc7-9ad5-4592-9892-a40db4ccf3d0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:00.712998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c489-fbcb-4e64-a1ad-c5b84b1a7ea3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:00.749698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:00.750450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.750686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.750971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.751831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:00.753335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dec970-d14f-43e0-af35-bd23609e2a5c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:00.759628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:00.760671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:00.761126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:00.761458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:00.762232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:00.765033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:00.765033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca5d4-4f3b-43cc-95bb-aea4f80ba8d8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:00.769155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 446/880] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:00.828418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 435/881] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:00 2022] POST /identity/v3/services => generated 269 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:01.375911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27487-a4b7-47b4-9efd-3ac384709f4b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.376469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27487-a4b7-47b4-9efd-3ac384709f4b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.376469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27487-a4b7-47b4-9efd-3ac384709f4b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.378089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 447/882] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:01.383466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5d73-4832-4b62-a86b-9b0afd479eba None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.383869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5d73-4832-4b62-a86b-9b0afd479eba None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.384035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5d73-4832-4b62-a86b-9b0afd479eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.401372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-227d5d73-4832-4b62-a86b-9b0afd479eba None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:01.402552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-227d5d73-4832-4b62-a86b-9b0afd479eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:01.475521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 436/883] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:01.498106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27487-a4b7-47b4-9efd-3ac384709f4b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:01.584646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609cae-8ad6-4d6d-831e-abd8b8c81f6d None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.584942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609cae-8ad6-4d6d-831e-abd8b8c81f6d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.585208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609cae-8ad6-4d6d-831e-abd8b8c81f6d None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.586808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 437/884] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:01.591721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.592010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.592285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.606058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27487-a4b7-47b4-9efd-3ac384709f4b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:01.609931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:01.611102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:01.648039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:01.648778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.649058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.649328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.650088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:01.651651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:01.653318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:01.665260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:01.670456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:01.671849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 448/885] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:01.673846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:01.700658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 438/886] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:01.720361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:01.735487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:01.797822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0bc5-c8d5-4408-a48c-a0a189613ef6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:01.833070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19fe93-f7e9-4d6a-a83c-06f800dc137e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:01.839698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:01.840476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.840884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.841673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.842493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:01.842939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:01.845312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:01.856649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 449/887] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:01.860109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:01.880618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:01.881574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:01.881976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:01.882355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:01.896473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:01.907404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:01.908534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:01.909522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:01.910167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 439/888] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:01.916193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:01.938284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:01.992857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb7f9-ef89-410f-84bc-86c41a69bceb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:01.994411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:02.039734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d848-d626-4313-95df-37928a1fc182 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.041834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.043680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.044167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.044663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.045576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:02.046178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.048314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.056164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 450/889] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:02.061429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.084941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.086716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.087236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.088261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.089650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:02.090636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.092677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.108417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 440/890] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:02.111993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.176697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd49c-687e-42f0-bf31-8f8c0b4b58d7 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.204916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:02.214310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.214976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.217000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.217000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.217000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] RBAC: Adding query filter params (interface=public, service_id=0884beec4073417abb1960ccae2bcf1f, region_id=RegionOne) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:02.217000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.218697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.224501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c5aee-f326-4472-b87a-c557202dfc16 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.228994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 451/891] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/endpoints?service_id=0884beec4073417abb1960ccae2bcf1f&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:02.229876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.262613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.263764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.264316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.264943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.267345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:02.277986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.279649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:02.300067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:02.301261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:02.302182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:02.302182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:02.302182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:02.302182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:02.308882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 441/892] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:02.310468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.341955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7985377-912b-4afb-9afe-b5faedc7c3a2 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.378374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.379142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.379517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.379800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.380486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.382031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.392870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 452/893] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:09:02.425417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4b45b-980a-451f-bb08-689f2aa8b2b5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.466659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.467696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.468278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.468813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.469825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:02.470330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.471929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.486822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 442/894] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:02.492831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.555979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:02.604865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917f88d-d7d0-416d-a6c0-d9e97ee7bc85 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.644815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.645634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.645920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.646193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.654282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.655807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:02.672078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:02.673098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:02.674026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 453/895] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:02.680694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.732506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:02.792352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46366-2eed-40ac-9204-21595ce9ff58 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.827654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:02.828323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:02.828615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:02.828895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:02.829731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:02.830034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:02.831455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2d18-dca3-40c1-8617-337a08a1b8ef None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:02.850051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 443/896] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:02.855546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:02.966000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859647-cf2f-4613-9334-c38772fb1559 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:02.980873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.002514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.003088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.003365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.003696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.074058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.075533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.078807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34136b-18da-4c14-9477-5bb0deb0275e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.079098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34136b-18da-4c14-9477-5bb0deb0275e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.079377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34136b-18da-4c14-9477-5bb0deb0275e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.081044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 444/897] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:03.086057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.086355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.086616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.103039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:03.104302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:03.176921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 445/898] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:03.196651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.197558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 454/899] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:02 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:03.202930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.217709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.240979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.306614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c4432-2455-4bcd-9477-95ddfb3b34a0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.306925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4860-b983-42fd-9b90-6d04dd8fd980 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.342731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.343578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.343863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.343962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.344135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.344709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.344950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.345226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.345953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.347307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b15e16-6183-4029-887a-5602f9917989 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.355032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.356488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-0f1b6d16-1794-45c1-bdc1-2ecdd9cbdd61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:03.374743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:03.375799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:03.376735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:03.376735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:03.376735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:03.376735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:03.376735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 455/900] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:03.377507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97473-daa6-4e9b-801e-c446fcbd6a93 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.377783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97473-daa6-4e9b-801e-c446fcbd6a93 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.378058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97473-daa6-4e9b-801e-c446fcbd6a93 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.379426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 456/901] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:03.381535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97473-daa6-4e9b-801e-c446fcbd6a93 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.429151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 446/902] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:03 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:03.429151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.429151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.429151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.452930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:03.454340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:03.460330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.507482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97473-daa6-4e9b-801e-c446fcbd6a93 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.552610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 447/903] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:03.555754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.556442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.556806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.557028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.557771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:03.558075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.559486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.573875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.580953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 457/904] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:03.582377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.612684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.715330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250d6e2-7cf1-4a50-aa38-ce9b284a4ea5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.715558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9186-7054-4e22-8c1a-e54a68d569db None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.753664 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.754854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.755413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.755975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.757022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.758813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.759091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.759621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.760151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.760428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.768547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.770048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.772922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:03.774120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:03.775034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:03.775899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:03.778850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 448/905] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:03.785527 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:03.786042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:03.786926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:03.787837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 458/906] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:03.794582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.890445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:03.903115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ea04a-d0e4-4d5a-97d7-0e40c08ca700 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.930943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e50fd-f645-4afc-924a-2c1cbb6219e0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:03.947939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.948630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.948923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.949202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.950162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:03.950489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.952797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.963838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 449/907] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:03.965690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:03.972234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:03.972934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:03.973218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:03.973494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:03.974281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:03.974588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:03.976023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:03.994151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 459/908] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:03.999982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:04.057318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:04.090720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5436f-22ef-43d8-b451-e472b636bee7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:04.121107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017fbc3-2dd0-4d34-9f59-138877fce4df None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:04.133141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:04.134369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.135113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.135865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.137277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:04.138080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:04.140934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:04.148252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 450/909] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:04.153394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:04.160090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:04.160797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.161085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.161360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.162201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] RBAC: Adding query filter params (role.id=c9f82684eac243558bd5051dd639d551, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=37821819ba394afda7a92b7aa99096b2) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:04.162514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:04.163959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:04.180611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 460/910] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:03 2022] GET /identity/v3/role_assignments?user.id=37821819ba394afda7a92b7aa99096b2&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=c9f82684eac243558bd5051dd639d551 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:04.181419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:04.273543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b96d-22df-49c1-80d1-f05e94b09b67 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:04.314171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:04.314844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.315128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.315403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.316210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] RBAC: Adding query filter params (interface=public, service_id=18a3bfff6c65472eadadcdbf66bca79e, region_id=RegionOne) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:04.316523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:04.318903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599511-0f08-410b-999e-ae2c8bbb448a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:04.326156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 451/911] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:09:04 2022] GET /identity/v3/endpoints?service_id=18a3bfff6c65472eadadcdbf66bca79e&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:04.333766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:04.450613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd1ea-b9a0-4104-8308-689d9520550b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:04.459273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:04.491034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:04.492043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.492532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.493049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.493936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:04.495513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d32ae-d3a2-418a-9c57-901e2d5466c6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:04.502784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 461/912] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:04 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:09:04.789311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964f2a-a76a-40f5-93b1-2c7c98424dc2 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.789810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964f2a-a76a-40f5-93b1-2c7c98424dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.789810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964f2a-a76a-40f5-93b1-2c7c98424dc2 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.791457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 452/913] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:04.797421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:04.797712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:04.797989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:04.815507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:04.817066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:04.890458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 462/914] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:04.912610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07964f2a-a76a-40f5-93b1-2c7c98424dc2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:05.000333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:05.022279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07964f2a-a76a-40f5-93b1-2c7c98424dc2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:05.058712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:05.059434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.059707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.059988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.060674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:05.062587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:05.074895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:05.076727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:05.078396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 453/915] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:05.083082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:05.187395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:05.190533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8398a23-7061-4887-a2a9-5ef8d2d7c3bc None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:05.225612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:05.226284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.226568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.226849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.227558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:05.227864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:05.229397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:05.234644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 463/916] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:05.241909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:05.349172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601dd2e-8fd1-4683-97aa-daa307a39631 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:05.387861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:05.388531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.388852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.389133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.390396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:05.390707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:05.392124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540cd8d-720c-46b6-b9a5-8f3abd94d4a0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:05.397624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 454/917] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Jun 7 09:09:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:05.404828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:05.512803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bacd47-9b8e-4109-995e-56b0b59e7653 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:05.547797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:05.548459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.548788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.549143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.549885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:05.551254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028dc3-3c60-410c-bdc8-369936a6382b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:05.641121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 464/918] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:05.821929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da49db-90bd-4e18-a205-0ef5b065c966 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.822452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da49db-90bd-4e18-a205-0ef5b065c966 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.822528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da49db-90bd-4e18-a205-0ef5b065c966 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.824158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 455/919] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:05.829768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:05.830062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:05.830321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:05.847976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:05.849202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:05.865680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:05.930296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 465/920] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:05.954237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da49db-90bd-4e18-a205-0ef5b065c966 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:05.966125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:06.061022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da49db-90bd-4e18-a205-0ef5b065c966 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.096710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.097516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.097795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.098066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.098720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:06.100221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:06.116605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:06.118703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:06.119567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 456/921] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:06.125899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.233970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08223f88-5394-4fc9-9eb5-29761f234ae2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.269585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.270258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.270546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.270827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.271547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:06.271853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:06.273258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:06.278516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 466/922] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:06.283849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.361281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:06.392610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6443a9-1b10-4b76-9734-e3a1286303dc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.426928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.427587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.427916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.428192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.428925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:06.429227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:06.430587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d906c73-dd2c-4d86-ad64-272d49df3d11 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:06.435739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 457/923] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:06.441675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.454798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:06.548085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b457f-ebca-4f47-ade9-45e24db2fc14 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.548384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b457f-ebca-4f47-ade9-45e24db2fc14 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.548567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b457f-ebca-4f47-ade9-45e24db2fc14 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.550203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 458/924] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:06.555088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.555380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.555655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.561086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bee71-8df6-4940-bb08-b431d20d393d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.572970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:06.574387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:06.601161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.601828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.602110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.602384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.603169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] RBAC: Adding query filter params (interface=public, service_id=56cfa458336e451c8b9435b164d4db98, region_id=RegionOne) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:06.603483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:06.604881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:06.611301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 467/925] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/endpoints?service_id=56cfa458336e451c8b9435b164d4db98&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:06.616072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.671573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 459/926] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:06.694663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.700531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:06.727567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249424e3-61ce-4626-b2d4-052a28da31ee None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.765661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.766766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.767368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.767788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.769080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:06.770873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed008c5-0bef-4d1d-b837-9d64ad140a1b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:06.775495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:06.789093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 468/927] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:09:06.820710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35305e96-27da-4fdc-ab91-4433e3c13b71 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:06.839156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb10ba-c1f5-4aea-8988-8f59d101bb4b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.839384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb10ba-c1f5-4aea-8988-8f59d101bb4b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.839639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb10ba-c1f5-4aea-8988-8f59d101bb4b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.841239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 469/928] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:06.847667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.847986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.848231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.868340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:06.869618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:06.871523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:06.874532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:06.875016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:06.875448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:06.892705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:06.904292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:06.906089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:06.907239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:06.916784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 460/929] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:06.919078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:06.970102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 470/930] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:06.993251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.054961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b8a8-c46e-4165-a173-af7d2acd4aeb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.081081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.083658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.092897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.093583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.093868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.094148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.094935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:07.095246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.096646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.111253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 461/931] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:07.112709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.119095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680853d-d32e-4266-a77d-c759105e00c0 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.155865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.157025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.157496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.157953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.158869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.160526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:07.173911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:07.174873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 09:09:07.175755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:07.183489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 471/932] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Jun 7 09:09:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:07.184624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.223871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968ebb-933c-4b90-9626-953a69eedeec None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.263546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.265112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.265509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.265885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.276756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.278290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:07.297149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:07.298293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:07.299393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:07.299393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:07.299393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:07.299393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:07.305234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 462/933] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:07.306631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.331333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62842ddf-095a-4c90-8cb6-0e42eb8166c4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.366813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.368613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.372403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.373286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.373719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.374663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.375662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:07.397101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.397101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.397101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 472/934] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:07.397101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.431783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276612f6-d691-4847-96cd-cc224f0605f8 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.469059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.469980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.470250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.470626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.471457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:07.471887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.473398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.490212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 463/935] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:07.491312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.497385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231753d-fde1-4e0a-8270-5992e8282b4d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.535693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.536772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.537205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.537626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.538594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:07.539046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.540575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.546773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 473/936] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:07.551905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.614579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.618761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0b40-c6d1-41c6-a145-cba61f2b36ad None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.628239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:07.665127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.666504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.667111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.667843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.679048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.680487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.683321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba290db-e4f3-476a-a263-1697f4701ea0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:07.704240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:07.705256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:07.706161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:07.708960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 464/937] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:07.714480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:07.726562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.727580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.728132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.728689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.729741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.731638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.826143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 474/938] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:07.836422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc44d4-bbfe-4672-96f1-aa12960dcfbd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:07.888210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:07.889944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:07.890487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:07.890971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:07.892052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:07.892637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:07.894422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3943722-b086-4a65-90d8-eba2df2c8384 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:07.916710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 465/939] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:07.925742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.048999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97c98-859d-428f-8a07-50865c5fa560 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.067024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.080219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676b14-e53a-4080-b388-2a1b0c227938 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.080510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676b14-e53a-4080-b388-2a1b0c227938 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.080827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676b14-e53a-4080-b388-2a1b0c227938 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.082320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 466/940] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:08.088375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.088699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.089124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.089590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.090412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.090692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.090968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.102851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.104330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.111446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:08.112711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:08.124001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:08.124974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:08.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:08.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:08.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:08.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:08.125892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 475/941] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:08.127157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.131331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.203542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 467/942] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:08.227453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.247870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ecd4-6757-43dc-9a12-2d373b107abe None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.290124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.291185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.291680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.292151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.293532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:08.294299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.296407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.312402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 476/943] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:08.318985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.352051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.357376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68529544-d7a8-4f53-a38a-c91f51c95634 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.399548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.402015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.402309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.402591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.403322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.404849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.406439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING keystone.server.flask.common [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 09:09:08.429099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.444898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1341bf-7f7e-4947-bef1-9d132ce06a7b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.458297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.sql.core [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 09:09:08.458297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 09:09:08.458297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [parameters: {'id': '950063d434c84f51904fba79bf0928bd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 09:09:08.458297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80906) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 950063d434c84f51904fba79bf0928bd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 950063d434c84f51904fba79bf0928bd. Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:09:08.468749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result.read() Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 09:09:08.469611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application yield resource Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise self.value Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application session.commit() Jun 07 09:09:08.470622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.session.flush() Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application compat.raise_( Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 09:09:08.471535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 09:09:08.472448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application result.read() Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:09:08.473292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [parameters: {'id': '950063d434c84f51904fba79bf0928bd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.474173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:08.474957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 950063d434c84f51904fba79bf0928bd. Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 468/944] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Jun 7 09:09:08 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:08.478242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.491964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.495043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.495043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.495043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.501623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.503491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:08.519591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:08.520552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:08.521534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:08.524053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 477/945] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:08.532259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.605119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb4310-34e0-4295-8e58-acfac7127589 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.642319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.643171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.643462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.643741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.645023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.650992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.652686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.658748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80f6d-74c8-47ea-b7e8-7de7715074a4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:08.668145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:08.669137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:08.670083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:08.674335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 469/946] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:08.675234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.702543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.703836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.704687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.705635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.706957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:08.707801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.710256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.733039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 478/947] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:08.737967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:08.744443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:08.795095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dba3b9-7d5d-44ff-bed8-6a19d7656bfc None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.846946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a537-313e-4902-9912-40823a357d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.848211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a537-313e-4902-9912-40823a357d56 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.848748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a537-313e-4902-9912-40823a357d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.849461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a537-313e-4902-9912-40823a357d56 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.850837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a537-313e-4902-9912-40823a357d56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:08.851605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a537-313e-4902-9912-40823a357d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.853513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a537-313e-4902-9912-40823a357d56 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.862073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74108ea-0e71-4834-ab21-05c1a8acd40a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:08.876412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 470/948] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:08.904126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:08.905495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:08.906002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:08.906643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/37821819ba394afda7a92b7aa99096b2/roles/c9f82684eac243558bd5051dd639d551` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:08.909904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=37821819ba394afda7a92b7aa99096b2, role_id=c9f82684eac243558bd5051dd639d551)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:08.911809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5633-8d68-4979-9c62-4afe70658a40 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:08.995333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 479/949] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:09:08 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/37821819ba394afda7a92b7aa99096b2/roles/c9f82684eac243558bd5051dd639d551 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:09:09.175042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011871b4-d7e4-4dc1-9305-7b8ca4e7fd5f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.175572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011871b4-d7e4-4dc1-9305-7b8ca4e7fd5f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.175572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011871b4-d7e4-4dc1-9305-7b8ca4e7fd5f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.177273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 471/950] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:09.183178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.183480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.183732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.202485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:09.203676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:09.205848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:09.274184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 480/951] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:09.296982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011871b4-d7e4-4dc1-9305-7b8ca4e7fd5f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:09.406850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011871b4-d7e4-4dc1-9305-7b8ca4e7fd5f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:09.427891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:09.442734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:09.443535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.443815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.444085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.444767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:09.446127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:09.458985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:09.460150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:09.461187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 472/952] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:09.466852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:09.574132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66141b65-6921-4fee-a56a-f4e9d6ec63ca None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:09.615304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:09.615974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.616258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.616536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.617283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:09.617591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:09.619032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:09.624978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 481/953] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:09.630511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:09.737760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e03c-d0ed-4746-843e-35d05e7e3759 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:09.774032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:09.774700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.774985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.775261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.775979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:09.776287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:09.777684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f80e6-d1df-466d-afdf-828810e2b136 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:09.783325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 473/954] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:09.788994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:09.791909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:09.900117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6803cf-decf-443e-9d73-a669e1f621ab None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:09.939200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:09.940234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:09.940781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:09.941280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:09.942217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:09.943800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05b981-7114-4c2e-8fc7-d38eb4dc54f4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:10.029283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 482/955] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:10.232395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3279c4-8cd2-40b4-b7b5-763fc49b1409 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:10.233085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3279c4-8cd2-40b4-b7b5-763fc49b1409 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:10.233085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3279c4-8cd2-40b4-b7b5-763fc49b1409 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:10.234657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 474/956] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:10.240226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:10.240523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:10.240834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:10.258506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:10.259956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:10.315494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:10.334588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 483/957] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:10.357205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3279c4-8cd2-40b4-b7b5-763fc49b1409 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:10.411217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:10.465041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3279c4-8cd2-40b4-b7b5-763fc49b1409 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:10.500056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:10.500808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:10.501092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:10.501368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:10.502017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:10.503373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:10.515471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:10.516449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:10.517333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 475/958] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:10.522545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:10.631525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2dbd9-7e3c-4898-b0fa-145a6ed6d477 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:10.667849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:10.668516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:10.668839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:10.669172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:10.669891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:10.670197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:10.672098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee227ec0-417c-437c-8599-3b5f9b7915f8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:10.677517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 484/959] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:10.682766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:10.792342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976e77e-bfa2-4388-bde9-4f022cab5f0f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:10.806658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:10.828848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:10.829568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:10.829859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:10.830137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:10.830911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:10.831294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:10.832743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deceac7-b275-4cf0-b34e-5c95d22106e1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:10.837858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 476/960] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:11.285520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2153-16ec-414d-99b1-e76c4a25703c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.286512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2153-16ec-414d-99b1-e76c4a25703c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.286987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2153-16ec-414d-99b1-e76c4a25703c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.288791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 485/961] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:11.294457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa138a-3fa9-4269-92bf-846ceb5781f4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.295300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa138a-3fa9-4269-92bf-846ceb5781f4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.295854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa138a-3fa9-4269-92bf-846ceb5781f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.313888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f8fa138a-3fa9-4269-92bf-846ceb5781f4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:11.315756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f8fa138a-3fa9-4269-92bf-846ceb5781f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:11.387637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 477/962] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:11.412117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2153-16ec-414d-99b1-e76c4a25703c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:11.441011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5ce4-238e-4f54-b59d-742ea12b3bd4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.441605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5ce4-238e-4f54-b59d-742ea12b3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.442022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5ce4-238e-4f54-b59d-742ea12b3bd4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.443953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 478/963] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:11.449210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.449496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.449771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.466499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:11.467739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:11.469975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:11.530680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2153-16ec-414d-99b1-e76c4a25703c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:11.548772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 479/964] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:11.567554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:11.568104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:11.568805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.569094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.569374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.570082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:11.571550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:11.572894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING keystone.server.flask.common [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 09:09:11.604114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:11.649760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.sql.core [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 09:09:11.649760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 09:09:11.649760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [parameters: {'user_id': 'ce75a67393974354b809428ba9c2d188', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 09:09:11.649760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80907) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application result.read() Jun 07 09:09:11.666495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:09:11.667576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application yield resource Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application next(self.gen) Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise self.value Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application session.commit() Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 09:09:11.669034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self.session.flush() Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application compat.raise_( Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 09:09:11.670158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 09:09:11.671919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application conn.query(q) Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application result.read() Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 09:09:11.678227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ce75a67393974354b809428ba9c2d188', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:11.679086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 09:09:11.679875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 486/965] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Jun 7 09:09:11 2022] POST /identity/v3/users => generated 160 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:11.680903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:11.684534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5779a3b-00f3-40f0-b59c-54ef8fe0fc8e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:11.739957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:11.741658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.742150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.742839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.757980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:11.768111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:11.769069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:11.775442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 480/966] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:11.777647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:11.808669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603568d-c9e1-4563-9676-e825c3acc049 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:11.823717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:11.858734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:11.859880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.860387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.860903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.872417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:11.875542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:11.884019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:11.892344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea584aab-7101-48b4-ba31-a6df6b94c514 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:11.897930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:11.898868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 09:09:11.899827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:11.906971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 487/967] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:11.908881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:11.934442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:11.935224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:11.935510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:11.935789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:11.937139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:11.937452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:11.938944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:11.953578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 481/968] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:11.955805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.043940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba0a5-0987-4e47-b88e-307e2586e969 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.086088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.086784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.087061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.087414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.088426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:12.088773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.090447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.093370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3dc89-843b-4acb-81e7-09f6f96e203c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.129996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.131158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.131651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.132163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.136608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 488/969] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Jun 7 09:09:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:12.151737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.153264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:12.173522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:12.174605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:12.176424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:12.176424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:12.176424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:12.176424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:12.176424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 482/970] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:12.185011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.225234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.304375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e58e-5904-4797-a025-0aa57e7b39ff None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.341471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.342060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.342341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.343230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.343956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:12.344260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.345667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3eb01-51a7-45d9-8972-c5e064875129 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.357818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 489/971] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:12.363506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.384057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.392411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d5042-11ff-49dc-a1e8-8ecc42b8a249 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.394004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d5042-11ff-49dc-a1e8-8ecc42b8a249 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.394004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d5042-11ff-49dc-a1e8-8ecc42b8a249 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.394510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 490/972] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:12.399864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.400160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.400445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.419414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:12.421004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:12.486035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e757-5fb3-489e-9f32-a291385fd524 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.493817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 491/973] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:12.516533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.523005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.523819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.524100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.524399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.533343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.534901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.535227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:12.551537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:12.557349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:12.558563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 483/974] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:12.561387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.628066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba549dc-ad84-4d52-98e9-e77cc980df6f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.637251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.666811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.667606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.667979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.668165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.668832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.670298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.672500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0701f-a350-4090-899c-6978a8433d74 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:12.682888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:12.683922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:12.685578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 492/975] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:12.690619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.711259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.711936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.712221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.712496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.713365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:12.713675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.715081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.736929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 484/976] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:12.742463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.781774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.800438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ce38-123c-44b9-88fb-3d35dba6691f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.837081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.837775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.838063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.838340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.839045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:12.839617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.840754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.848794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb016-e3d4-48f6-aedf-6f24b4e5f3d3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.850806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 493/977] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:12.851803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:12.881552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:12.891101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:12.891757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:12.892044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:12.897697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:12.963620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e81-9ddf-4c07-9df9-19e16c161a42 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:12.966497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:12.967937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:12.999662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.000350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.000661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.000942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.001651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:13.001960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.005598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.010747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 494/978] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:13.017164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.030285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.079048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 485/979] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 09:09:13.084441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.114043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.125214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47983ca-6dad-47b9-9f67-4fcca559089b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.160436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.161036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.161316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.161588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.162381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] RBAC: Adding query filter params (interface=admin, service_id=56cfa458336e451c8b9435b164d4db98, region_id=RegionOne) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:13.162690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.164047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.169342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 495/980] 10.222.0.57 () {56 vars in 1249 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/endpoints?service_id=56cfa458336e451c8b9435b164d4db98&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:13.174944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.193004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5643c-a003-4cb2-9632-07a66c86b9d6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.227506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.228288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.228564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.228867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.238627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.240034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.250120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-755a617e-7a89-455e-b0e4-eebfd7546354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:13.258436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:13.264418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 486/981] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495fb42-545d-49d4-961d-dad6dcdd5263 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495fb42-545d-49d4-961d-dad6dcdd5263 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495fb42-545d-49d4-961d-dad6dcdd5263 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.266095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 487/982] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:13.267810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495fb42-545d-49d4-961d-dad6dcdd5263 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.276651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5e35-35d8-4a0a-b491-4441344146da None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.309738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.310392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.310670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.310943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.312155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.313532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c863a6-c792-4e94-a6e8-8d211a0afffd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.319372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 496/983] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:09:13.320503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.320815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.321088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.340071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:13.341321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:13.362293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.385048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495fb42-545d-49d4-961d-dad6dcdd5263 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.421353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 497/984] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:13.429119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.429769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.430145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.430517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.431623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:13.432034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.433587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.444373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.447408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 488/985] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:13.452851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.523764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.562860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c773e93-d4bb-44e1-87f8-38be00b18885 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.568544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eacb6-817b-47e0-a54c-419648e05682 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.598159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.598986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.599382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.599720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.600528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.602104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.604439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.605253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.605535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.605802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.613192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.614599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.617074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:13.629174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:13.630278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 489/986] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:13.636362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.657060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 498/987] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:13 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:13.749346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ff06-ae20-4dbe-a299-c3aa5985e03a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.793624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.794423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.794710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.794992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.796359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:13.796703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.798170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adc365-cdfe-442b-91f6-728c0d52838f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.816822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 490/988] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:13.818309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:13.928990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31f63-2ce6-4d04-8e29-2d61d18073fb None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:13.956672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:13.964394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] RBAC: Adding query filter params (role.id=c9f82684eac243558bd5051dd639d551, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=37821819ba394afda7a92b7aa99096b2) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:13.970648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a83f8a-2bef-4c30-a83d-ec63c01e2d84 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:13.975756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 499/989] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:13 2022] GET /identity/v3/role_assignments?user.id=37821819ba394afda7a92b7aa99096b2&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=c9f82684eac243558bd5051dd639d551 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:15.575141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13991f-0c93-4b34-9870-6c36968aede7 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:15.575707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13991f-0c93-4b34-9870-6c36968aede7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:15.575707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13991f-0c93-4b34-9870-6c36968aede7 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:15.577267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 491/990] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:15.582668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:15.583175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:15.583249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:15.602008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:15.603232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:15.680763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 500/991] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:15.699363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13991f-0c93-4b34-9870-6c36968aede7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:15.762434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:15.808417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13991f-0c93-4b34-9870-6c36968aede7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:15.844187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:15.844948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:15.845236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:15.845512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:15.846166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:15.847531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-e9883b81-c5fc-4f6d-8abb-386b583af951 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:15.859858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:15.860935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 09:09:15.861789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:15.863160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 492/992] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Jun 7 09:09:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:15.867176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:15.930471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c99e-dd5b-4bd4-8b31-06ebcd1014d4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:15.930822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c99e-dd5b-4bd4-8b31-06ebcd1014d4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:15.931030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c99e-dd5b-4bd4-8b31-06ebcd1014d4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:15.936424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 493/993] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:15.937694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:15.937986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:15.938266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:15.956334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:15.957555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:15.974207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0688-14af-4564-8fa6-31b263de0a4d None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:15.997453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.012931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.013608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.013891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.014703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.015477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.015783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.017306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.022289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 501/994] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:16.027882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.039158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 494/995] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:16.061689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.119608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.141560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae88bd-abe3-45ac-a0c6-938967dc33d0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.173919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ed2ae-a50d-422e-9ccc-e614c184de69 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.179455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.180151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.180437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.180747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.181460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.181766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.183156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.189162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 502/996] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:16.194671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.211766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.212721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.213061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.213343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.213996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.215361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:16.228248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:16.229297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:16.230151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:16.235115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 495/997] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:16.235960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.249438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.301164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b378c-4c21-4bb3-a238-89490df2297a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.334078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.334726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.334839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b469a-55ed-4704-b740-ae7f4624d18f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.335050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.335403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.336147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.337550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a745049-b30b-47ac-95bc-67e63e2892b5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.349189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.369065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.369820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.370000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.370269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.370966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.371270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.372659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ccae8-9526-4e87-9ddb-72fa22bf8e13 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.377323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 496/998] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:16.378531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee31b-b91e-4eb1-b74e-2125724974e7 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.378815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee31b-b91e-4eb1-b74e-2125724974e7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.379087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee31b-b91e-4eb1-b74e-2125724974e7 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.380362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 497/999] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:16.383116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ee31b-b91e-4eb1-b74e-2125724974e7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.412685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 503/1000] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:16.412685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.412685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.412685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.419669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:16.421543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:16.506801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ee31b-b91e-4eb1-b74e-2125724974e7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.508629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 504/1001] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:16.533343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.556970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.557654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.558416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.558731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.559470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.559783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.561280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.571675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 498/1002] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:16.573692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.579821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.626395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.652163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39757ae1-e28c-4d74-ac8a-91f19ba57cff None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.687279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.688088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.688369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.688688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] PATH_INFO: `/v3/services/image` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.689334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.690797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.691659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fae685-3117-4ee8-8735-9959a9eb9aa4 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:16.702843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:16.704184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:16.705055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 505/1003] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:16.710091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.729271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.730019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.730307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.730584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.731375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] RBAC: Adding query filter params (interface=public, service_id=33397453aea343a0ab7d533289a48cda, region_id=RegionOne) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.731686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.733598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.738855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 499/1004] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/endpoints?service_id=33397453aea343a0ab7d533289a48cda&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:16.744064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.815734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.819097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7549b87-b665-4929-9aa9-97773b010a6e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.851315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b06dea-6fb7-4277-a833-a59f0ff03f94 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:16.853771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.854439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.854720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.855053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.855827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] RBAC: Adding query filter params (name=image) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:16.856135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.857667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.859012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:16.864529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 506/1005] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:16.869954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:16.887114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:16.887704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:16.887993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:16.888264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:16.888992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:16.890965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3409-fd4a-477b-a1f0-0d9cc0c00601 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:16.897919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 500/1006] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:09:16.985619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0204-01e8-41bf-a0ee-c62d6bf9495a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:17.032299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] RBAC: Adding query filter params (type=image) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:17.045585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c734b5-37a3-4325-8c78-f15c5dea7337 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:17.046129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 507/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:18.668002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e568fd-760f-45e6-b251-e38f2f1868d8 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:18.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e568fd-760f-45e6-b251-e38f2f1868d8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:18.668560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e568fd-760f-45e6-b251-e38f2f1868d8 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:18.669992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 501/1008] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:18.675372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:18.675800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:18.675967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:18.693319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:18.694494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:18.718743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:18.763918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 508/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:18.788342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e568fd-760f-45e6-b251-e38f2f1868d8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:18.896220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e568fd-760f-45e6-b251-e38f2f1868d8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:18.930160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:18.930881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:18.931165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:18.931445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:18.932748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:18.934234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b33eb5-8f86-474c-ae89-533125bc1cf5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:18.936827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 502/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:18.942774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.058673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a9edb-2c18-4f12-97af-c225292cfe5b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.093524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.094184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.094467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.094801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.095511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.096974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963dd7-d3f0-4216-a34d-0e513b793116 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:19.185127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307cbc3-36ec-41f6-879f-76367913740f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.185344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307cbc3-36ec-41f6-879f-76367913740f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.185611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307cbc3-36ec-41f6-879f-76367913740f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.187035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 503/1011] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:19.191971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.192235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.192515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.209581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:19.210858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:19.218780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 509/1012] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:18 2022] POST /identity/v3/projects => generated 311 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:19.249992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:19.281409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03898-094b-41fa-a4b0-9c46c2e5703a None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.281699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03898-094b-41fa-a4b0-9c46c2e5703a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.281976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03898-094b-41fa-a4b0-9c46c2e5703a None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.283596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 510/1013] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:19.289171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.289466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.289741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.293413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 504/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:19.307271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:19.308478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:19.312387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:19.317022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.390266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 511/1015] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:19.413414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.439276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354820b3-166b-44e1-98e5-d6658179bc07 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.482667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.483561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.483840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.484110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.484805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.486192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:19.498953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:19.500012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:19.500907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 505/1016] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:19.505988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.530971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273524-59a9-431a-a3c6-d58f3cd0d36b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.539835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:19.565803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.566470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.566753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.567026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.567744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.569162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:19.613339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc53c-c7d7-4c33-b234-0e41159947bd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.647909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.648564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.648930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.649209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.649916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:19.650226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.651584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:19.652249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:19.657454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 506/1017] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:19.668632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 512/1018] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:19 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:19.668852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.778154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:19.786658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179483b-0e2a-4adf-86b3-bfb43fd65c71 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.831977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.832684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.832970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.833243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.833962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:19.834267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.835657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f318e-a046-4930-a938-6731d2594cf4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:19.846305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 507/1019] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Jun 7 09:09:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:19.847331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:19.952827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880f7a-b489-43bf-a9db-fc842ee555bc None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:19.987405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:19.988513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:19.988956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:19.989401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:19.990283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:19.991898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03848590-38f6-4dbb-8e1f-638e4f2af691 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:20.076938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 513/1020] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:21.557654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a813c9-67ff-499c-9a9b-7d09a149f615 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.558181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a813c9-67ff-499c-9a9b-7d09a149f615 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.558262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a813c9-67ff-499c-9a9b-7d09a149f615 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.559621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 508/1021] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:21.564981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.565392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.565552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.582505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:21.583733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:21.640610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:21.663747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 514/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:21.689353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a813c9-67ff-499c-9a9b-7d09a149f615 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:21.794035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a813c9-67ff-499c-9a9b-7d09a149f615 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:21.829059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:21.829770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.830055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.830331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.831515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:21.832975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cae11c-a6ea-43cd-ba10-af1798354d76 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:21.839254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 509/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:21.840421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:21.935932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e582d0-44b5-40fa-9238-24e8a351ff0a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.936224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e582d0-44b5-40fa-9238-24e8a351ff0a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.936499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e582d0-44b5-40fa-9238-24e8a351ff0a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.941647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 510/1024] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:21.942937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.943225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.943502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.945963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670cde6-a972-4035-a942-8064cdc1bff6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:21.960561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:21.961767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:21.981395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:21.982369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:21.983097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:21.983381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:21.983656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:21.984372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:21.985861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.034215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 511/1025] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:22.058735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.109565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 515/1026] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:09:21 2022] POST /identity/v3/users => generated 285 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:22.178048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc531f-110d-4a08-aab6-1a1c948a96d5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.216468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.217838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.218120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.218392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] PATH_INFO: `/v3/services/image` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.219057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.220443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:22.233542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:22.234705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:22.235551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:22.240019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 512/1027] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:22.241364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.286517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:22.349211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c135e0-8ef7-4ee8-8cff-9bf7e4b0ef57 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.385277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.385857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.386134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.386402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.387107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] RBAC: Adding query filter params (name=image) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:22.387407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.388794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00523fdf-9fbd-49b9-a6e7-aa5368cf411d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.394244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 516/1028] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:22.399921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.421471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:22.423396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9a61-f428-4e29-b6c2-9c150a8941bb None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.423683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9a61-f428-4e29-b6c2-9c150a8941bb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.423960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9a61-f428-4e29-b6c2-9c150a8941bb None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.426090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 517/1029] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:22.431082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.431374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.436172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.448851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:22.450060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:22.512952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d276651-931d-4113-bfae-c88d06c8d624 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.520698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 518/1030] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:22.548330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.554009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.554658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.554939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.555212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.555926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] RBAC: Adding query filter params (type=image) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:22.556234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.557624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.562963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 513/1031] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:22.570195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.585865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:22.657081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eac8d-d85c-4827-a5ef-8db8f1229149 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.658979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:22.677586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd6650-2750-4517-a4a2-6796ae5f591d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.692671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.693491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.693774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.694044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.705598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:22.713671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:22.713825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:22.714707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:22.715583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:22.716224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.716224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.716224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.716401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 519/1032] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:22.721086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] RBAC: Adding query filter params (interface=public, service_id=ca273844ed7842a68402f828bd0beb26, region_id=RegionOne) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:22.721417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.722449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.722788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.732837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 514/1033] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/endpoints?service_id=ca273844ed7842a68402f828bd0beb26&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:22.733781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:22.821120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:22.830555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3457b-ef73-4948-baf6-e9ba3f7bcbef None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.839692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7418d3-7a23-4fa0-8b9a-25746451f25b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:22.867597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.868263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.868546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.868847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.869710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:22.870232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.871603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.874865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:22.875530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:22.875815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:22.876092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:22.876821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:22.878159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:22.880314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 520/1034] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:22.892620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 515/1035] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:09:22.892851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:23.003430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4cad7-7857-4cc2-a98e-c27ca353ffae None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:23.046403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:23.047213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:23.047501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:23.047776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:23.059958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:23.062049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:23.080757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:23.081735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:23.083120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:23.083120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:23.083120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:23.083120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:23.083120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 521/1036] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:23.088036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:23.102262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:23.192281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d3f4-939a-4ec6-8e9a-ce2dc5f8fb6e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:23.226282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:23.226941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:23.227224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:23.227499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:23.228217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:23.228523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:23.229900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:23.245704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 516/1037] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:23.247255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:23.257238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:23.350059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9001b-e6bc-4971-8a25-f054d53fe526 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:23.385799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:23.386596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:23.386888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:23.387164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:23.395167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:23.396659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:23.412729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:23.413707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:23.414706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 522/1038] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:09:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:23.419742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:23.494880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:23.527384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2376-f007-4b06-b31f-44465669dfbd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:23.562569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:23.563239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:23.563772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:23.563772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:23.564626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:23.564939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:23.566380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:23.589377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 517/1039] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:09:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:23.591310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:23.659581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:23.694822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bafe21-97a5-4c24-92a0-4187e2beaf95 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:23.729407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:23.730060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:23.730340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:23.730614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:23.798691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:23.800059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:23.863422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:23.899221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 523/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 09:09:23.905373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.016055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372a8f-f780-4d01-8c91-5cccca549b8f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.043384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:24.052714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.053440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.053717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.054162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.064240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.065696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:24.084260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:24.085404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:24.086518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:24.086518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:24.086518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:24.086518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:24.089596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 518/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:24.094799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.197676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed918-71d2-438d-a343-44d6141d76b5 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.230814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.231463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.231740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.232009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.233248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:24.233552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.234910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.246037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 524/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:24.252041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.349766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12829f06-99c9-4846-8d5b-24116fecac59 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.383670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.384741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.385021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.385289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.392007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.393457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-6a9907ff-3252-43c6-869c-f676c1acb439 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:24.406798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:24.407676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:24.408574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:24.412658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 519/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:24.414062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.441863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:24.455212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d825-4ba5-4c69-ba20-5628f560f32c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.455566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d825-4ba5-4c69-ba20-5628f560f32c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.462562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d825-4ba5-4c69-ba20-5628f560f32c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.462562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 520/1044] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:24.462562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.463043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.463478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.481628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:24.482825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:24.518871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b0680-8581-4574-89c5-a8aea44004bd None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.523780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:24.553167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.553820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.554115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.554418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.555253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:24.555562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.557129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.559801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 521/1045] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:24.573885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 525/1046] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:24.580631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.583678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.670347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:24.684996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd49bc-ac5d-40c6-83e4-32df7ed9c132 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.688621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b29e-df8c-4d17-b327-5271affd7834 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.721586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.722344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.722625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.722942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.724107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.724776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.725057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.725329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.726150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] RBAC: Adding query filter params (role.id=c9f82684eac243558bd5051dd639d551, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:24.726455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.727868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.743975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 522/1047] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=c9f82684eac243558bd5051dd639d551 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:24.744194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:24.745021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:24.746006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 526/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:24.749685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:24.787162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:24.865097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61480e8c-20d8-41e2-9e3b-1a5ecac0fe74 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:24.902754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:24.903431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:24.903709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:24.903975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:24.904888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:24.905188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:24.906568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:24.915894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 523/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:24.922439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.034778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cd82c-ddec-455a-b1c7-12ca9fef91c9 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.069063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.069941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.070216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.070487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] PATH_INFO: `/v3/users/042d699b41d64e67917ea892d9416cc8` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.083818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] RBAC: Authorizing `identity:get_user(user_id=042d699b41d64e67917ea892d9416cc8)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.085308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998909c-87f4-4c8b-acbd-78bd208e6a3c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.087153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 527/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:24 2022] GET /identity/v3/users/042d699b41d64e67917ea892d9416cc8 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:25.092682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.154585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0603d1-17eb-4e7e-9ae1-5b86cab19b41 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.154875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0603d1-17eb-4e7e-9ae1-5b86cab19b41 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.155156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0603d1-17eb-4e7e-9ae1-5b86cab19b41 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.156686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 528/1051] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:25.161343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.161632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.161914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.167811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.185435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:25.185435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:25.189453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.195740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a635-80e1-483a-99f8-e307d750ff5a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.231504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.233106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.233589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.234169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.252327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 529/1052] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:25.275200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.307124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.308611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.378684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0479-5681-4c35-a092-35b06e38a261 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.401483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.408986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 524/1053] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:25.414793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.421491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.422285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.422567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.422842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] PATH_INFO: `/v3/services/image` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.423501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.424891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.426844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:25.445133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:25.446153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:25.447020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 530/1054] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:25.447823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.518238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4934-960b-49d2-9b46-44dfe0c1cae3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.555360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.556209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.556486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.556803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.558013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.559432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.560157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ed19d-02b8-4d83-989d-b045e6e3d001 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.561346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 525/1055] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:25.566998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.598412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.599078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.599615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.599896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.601305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] RBAC: Adding query filter params (name=image) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:25.601620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.603236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.612757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 531/1056] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:25.615118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.638419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.679319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.682130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28240-5db3-4302-8d50-3a05a530bd89 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.718127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.718698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.718980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.719251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.719539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e58440-1dad-426e-abb9-d39fafe80f06 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.720071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3, user.id=042d699b41d64e67917ea892d9416cc8) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:25.720382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.721817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.727566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 526/1057] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/role_assignments?user.id=042d699b41d64e67917ea892d9416cc8&scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:25.767001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.767680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.767961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.768233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.768988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] RBAC: Adding query filter params (type=image) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:25.769294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.770763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da795e2a-fd83-4839-8d09-f3477b96bcee None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:25.781879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 532/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:25.782903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:25.893545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec5b25-26b5-49d6-9e44-3b48d44ed720 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:25.929137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:25.937804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:25.938463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:25.938737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:25.939039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:25.939789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:25.941198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faa875-a640-4b79-a0bc-1fcce3c401b0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:26.159588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 527/1059] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:09:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:09:27.001371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f20f2-5cd7-4a70-8dd0-7dd3e23ffe1b None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.001932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f20f2-5cd7-4a70-8dd0-7dd3e23ffe1b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.001932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f20f2-5cd7-4a70-8dd0-7dd3e23ffe1b None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.004654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 533/1060] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:27.009623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.010019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.010199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.031717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:27.032928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:27.108627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 528/1061] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 07 09:09:27.129982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f20f2-5cd7-4a70-8dd0-7dd3e23ffe1b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.232912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f20f2-5cd7-4a70-8dd0-7dd3e23ffe1b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:27.257708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:27.267239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:27.273347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.273347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.273347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.280506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:27.289167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:27.290576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:27.291711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:27.296955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 534/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:27.300508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.402105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664b94c-1959-4f61-aaf3-b7ed2fab92d0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:27.413521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:27.436949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:27.437663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.437979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.438254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.438992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:27.439296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:27.440672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:27.449466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 529/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:27.457790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.557908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522d513-00fa-4086-b7b1-d5e3e8848d33 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:27.592276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:27.593102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.593384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.593655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.603542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:27.605025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:27.622641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:27.623583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:27.629531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:27.629531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:27.629531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:27.629531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:27.629531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 535/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:27.629839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.734343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b3-7b6d-43ba-aa0d-d71383903751 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:27.767739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:27.768399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.768718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.768994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.769712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:27.770016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:27.771371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:27.788077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 530/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:27.789313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.830202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:27.892169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49bda-b727-435d-96d8-89eaacaf3568 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:27.927235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:27.928391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.928907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.929399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.937222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:27.938917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-eb81bba6-919c-404d-a0b0-870fd31657ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:27.953887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:27.954869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:27.955841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:27.958145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 536/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:27.964026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:27.980689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977859ad-4907-4e5a-8b1b-5bbcacc1b289 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.981029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977859ad-4907-4e5a-8b1b-5bbcacc1b289 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.981770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977859ad-4907-4e5a-8b1b-5bbcacc1b289 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.983461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 537/1067] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:27.989209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:27.989636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:27.990059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:27.998687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.008791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:28.025309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:28.076746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86ede7-0cc3-450c-b226-6df823fc578f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.090000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 538/1068] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:28.105421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.114775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.115502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.115905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.116207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.117098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:28.117405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.118962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.135046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.146268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 531/1069] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:09:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:28.147743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.213850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06350e-2e8a-4d51-9d1a-7a0fa14439c5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.246461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.251959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.252790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.253072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.253468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.257537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23565e-aeb0-44a3-aaf2-3b460038a188 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.264351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:28.272015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:28.272987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:28.273859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 539/1070] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:28.279983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.298443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.299626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.299909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.300184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.313760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.315231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:28.333252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:28.334193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:28.335140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:28.335140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:28.335140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:28.335140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:28.335140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 532/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:28.340220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.377478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.391503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9ae0-c305-4d49-be0b-6f51ff9b982c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.430802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.431650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.431933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.432210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.432994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:28.433302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.434669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.443042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 540/1072] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:28.448137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd0bc3-2ec3-4e1c-8c39-b1b02ecb1f2a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.448925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.483279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.483960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.484240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.484512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.485267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:28.485570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.486916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40e6fd-8ada-4e00-aa90-d13951d75e8e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.497910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 533/1073] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:28.499625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147dcf-30c4-415f-bacf-a5acc2739452 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.499908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147dcf-30c4-415f-bacf-a5acc2739452 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.500179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147dcf-30c4-415f-bacf-a5acc2739452 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.501589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 534/1074] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:28.505562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147dcf-30c4-415f-bacf-a5acc2739452 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.508181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.548991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa04c8-4fae-406f-8c69-f2b77d5beaa0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.581720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.582595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.582889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.583156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] PATH_INFO: `/v3/users/042d699b41d64e67917ea892d9416cc8` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.584353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] RBAC: Authorizing `identity:get_user(user_id=042d699b41d64e67917ea892d9416cc8)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.585818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee8bac-a4cf-4d83-92ad-bb10af28be4c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.587436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 541/1075] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/users/042d699b41d64e67917ea892d9416cc8 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:28.588793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.589079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.589353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.605088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:28.606199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:28.611460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147dcf-30c4-415f-bacf-a5acc2739452 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.619301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.645440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.646223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.646506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.646776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.653538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.655043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:28.668400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:28.669388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:28.670812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 535/1076] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:28.671329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.673720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 542/1077] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:28.683156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.734829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.773945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d793-6b1d-4366-9379-9fd60a694233 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.782477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41ff2b-7c58-4e44-a3ac-28ebe82f1c83 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.807583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.808436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.808757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.809040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.810212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.812210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.814008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 536/1078] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:28.814764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.816574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.817252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.817531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.817803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.818577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:28.818885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.820289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:28.831775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 543/1079] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:28.832421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:28.844398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.913691 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a617f6-8797-4cca-8329-6ae7b31a93e6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.932159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c64d2-1bb9-4b71-bea8-2736f3523ee4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:28.941761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:28.947541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.953178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.953178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.953178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.965578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:28.966453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:28.967248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:28.967524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3/users/042d699b41d64e67917ea892d9416cc8/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:28.969508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] RBAC: Authorizing `identity:create_grant(project_id=84caeb95e5f64d12be326bc4c25d7cd3, user_id=042d699b41d64e67917ea892d9416cc8, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:28.970924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:29.018004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:29.019455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:29.022824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 544/1080] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:09:28 2022] PUT /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3/users/042d699b41d64e67917ea892d9416cc8/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:09:29.023731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:29.112739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:29.143840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 537/1081] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:29.149921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:29.150130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4e86a-3e61-4a0b-87d0-a3ef0084e7cc None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:29.183354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:29.190523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:29.191428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:29.191708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:29.194175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/c9f82684eac243558bd5051dd639d551` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:29.194366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=dff6b630f6be464aa6cdcc73da587e4f, role_id=c9f82684eac243558bd5051dd639d551)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:29.195804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:29.245227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 545/1082] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:09:29 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/dff6b630f6be464aa6cdcc73da587e4f/roles/c9f82684eac243558bd5051dd639d551 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:09:29.262314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901a002-2a82-4c3c-8055-66dc51fa199c None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:29.302289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:29.303091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:29.303376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:29.303653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:29.315433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:29.316985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-afb1db12-ae1b-420b-993e-392e0178b39e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:29.336701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:29.337807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:29.339000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:29.339000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:29.339000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:29.339000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:29.339000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 538/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:29.345967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:29.458093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1da9c-783f-418e-99f3-24d82b78b643 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:29.486177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:29.494321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:29.495265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:29.495767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:29.496243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:29.497199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:29.497709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:29.499289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73ac6-7381-496c-beda-dd22dfd9d94c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:29.511876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 546/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:31.260908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35b6ff-d9f1-4d6e-82a5-a8d0dc563982 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.261414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35b6ff-d9f1-4d6e-82a5-a8d0dc563982 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.261414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35b6ff-d9f1-4d6e-82a5-a8d0dc563982 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.262750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 539/1085] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:31.268022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6998-0010-4b72-b450-0925bf60fd42 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.268412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6998-0010-4b72-b450-0925bf60fd42 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.268719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6998-0010-4b72-b450-0925bf60fd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.285228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8c8c6998-0010-4b72-b450-0925bf60fd42 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:31.286391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8c8c6998-0010-4b72-b450-0925bf60fd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:31.350401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 547/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:31.373822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35b6ff-d9f1-4d6e-82a5-a8d0dc563982 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.454110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:31.472633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35b6ff-d9f1-4d6e-82a5-a8d0dc563982 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.505979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.506757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.507036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.507303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.511553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64783-f71d-4d8a-9b5b-08274f7b80b8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.511857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64783-f71d-4d8a-9b5b-08274f7b80b8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.512207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64783-f71d-4d8a-9b5b-08274f7b80b8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.514226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 548/1087] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:09:31.518901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:31.519036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.519248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.519523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:31.526578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:31.527567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:31.529011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 540/1088] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:31.533537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.535850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:31.537070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:31.601184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 549/1089] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:31.625405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.632128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95904b-1850-4e5a-b38a-55d22e7a5dfa None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.664838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.665490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.665772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.666044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.666787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:31.667089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:31.668443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:31.672611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:31.676893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 541/1090] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:31.682453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.686730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:31.723168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63c207-d52c-4470-a16c-cfa1b936e2f6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.757607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.758534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.758821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.759359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.769949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:31.777987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:31.778932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:31.779972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 550/1091] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:31.784194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955f9f-7edc-4575-b66a-91a1363960c3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.785077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.818102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.818968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.819251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.819526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] PATH_INFO: `/v3/users/042d699b41d64e67917ea892d9416cc8` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.820757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] RBAC: Authorizing `identity:get_user(user_id=042d699b41d64e67917ea892d9416cc8)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:31.822180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:31.823813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 542/1092] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/users/042d699b41d64e67917ea892d9416cc8 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:31.829885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.885425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b38e1-27c2-4787-8a06-147eb133a0e8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.894715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:31.899366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:31.919861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.920525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.920873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.921152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:31.921898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:31.922203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:31.923592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:31.929471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc7837-4bc4-4ca5-a638-a5fd2ae8e470 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:31.931597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 551/1093] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:31.937232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:31.964459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:31.965126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:31.965416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:31.965689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.051523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.051702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7499292-8ad0-4300-855d-e5ce16c2c8b4 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.051797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.071814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.072622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.072908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.073185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.084671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.086144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:32.103356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:32.106635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:32.107651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:32.107651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:32.107651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:32.107651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:32.107651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 552/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:32.108766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.110326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.135859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.141031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 543/1095] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 09:09:32.147048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.211622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff9b8-0fd8-41d9-ad9f-7b2e38daa5ce None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.245069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a304890-b46d-423d-9980-51e8e3f4d4e3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.245239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.245834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.246119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.246400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.247115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:32.247426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.249361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.261455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 553/1096] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:32.267723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.280395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.281286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.283688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.283688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.283688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.284444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.286175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 544/1097] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:32.292196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.333951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.350319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.368521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c78f-90a6-4153-8a18-98915844c94b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.392054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10435231-52d6-43c7-982d-8001655642a1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.403098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.403886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.404786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.405064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.412589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.414054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.425921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.426561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.426845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.427121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.427945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3, user.id=042d699b41d64e67917ea892d9416cc8) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:09:32.428108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:32.429116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 09:09:32.430056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:32.430595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.430672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 554/1098] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:09:32.430742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.435051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.435858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 545/1099] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/role_assignments?user.id=042d699b41d64e67917ea892d9416cc8&scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:32.540474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037df2a2-d0a2-4c6f-b96f-725fac201b48 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.575739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.576966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.577254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.577545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.578424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:32.578737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.580203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.596105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.600049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 555/1100] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:32.605974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:32.710282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83b5cd-172b-4f59-9dca-0730fac12257 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:32.712617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:32.743564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:32.744210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:32.744493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:32.744806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:32.811411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:32.812951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:32.910796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 546/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:32.916453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:33.012300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bca58-00a7-4685-b7fb-03986ca27600 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:33.048117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:33.048974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:33.049248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:33.049518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:33.059288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:33.061288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:09:33.078755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:33.079652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:33.080555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:09:33.080555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:09:33.080555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:09:33.080555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:33.080555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 556/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:33.085617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:33.088733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:33.181814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eab2f-4204-4ff1-b695-9b353a1864c3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:33.214982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:33.215636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:33.215916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:33.216186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:33.216961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:33.217265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:33.218626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:33.229566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 547/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:33.235790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:33.287948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:33.334098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d846c8-3baa-4970-9d8a-b7c7b1667a44 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:33.366517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:33.367294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:33.367572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:33.367843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:33.374865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:33.376294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:09:33.389906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:33.390852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:33.391760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 557/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:09:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:33.396876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:33.456988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:33.493479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efab35-644a-4878-9880-ff578b85ca30 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:33.525744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:33.526386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:33.526666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:33.526937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:33.527707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:33.528013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:33.529984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85571ff-e645-40c1-80b6-c3c387c2525e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:33.541592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 548/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:09:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:33.548075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:33.646918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4025c7e-3756-49e4-882c-8aeb149600e6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:33.659979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:33.680687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:33.681333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:33.681614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:33.681888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:33.682705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] RBAC: Adding query filter params (role.id=c9f82684eac243558bd5051dd639d551, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=dff6b630f6be464aa6cdcc73da587e4f) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:33.683013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:33.684436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016347ee-2f10-4106-b10c-6bf6e5ec24e6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:33.690185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 558/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:33 2022] GET /identity/v3/role_assignments?user.id=dff6b630f6be464aa6cdcc73da587e4f&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=c9f82684eac243558bd5051dd639d551 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:34.713210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8820-f977-45d4-8683-766eee891052 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:34.713775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8820-f977-45d4-8683-766eee891052 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:34.713775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8820-f977-45d4-8683-766eee891052 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:34.715109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 549/1107] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:34.720256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:34.720700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:34.720844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:34.737645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:34.738830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:34.802711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 559/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:34.827291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee8820-f977-45d4-8683-766eee891052 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:34.925333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee8820-f977-45d4-8683-766eee891052 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:34.948243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:34.958404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:34.959172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:34.959455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:34.959729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:34.960920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:34.962331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b3112-6636-4511-afed-fc0138cb36fc None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:34.971490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 550/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:34.971635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:35.066869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57143a-334d-45da-b047-b1e7f8c46ea5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:35.073842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:35.099897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:35.100627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:35.101134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:35.101588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:35.102589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:35.104024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd7507-7ccc-4ae8-8525-22a36f40289d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:35.192294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 560/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:09:34 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:37.441272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc91e-ffba-4627-9b4e-3044c6c6896f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:37.441790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc91e-ffba-4627-9b4e-3044c6c6896f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:37.441883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc91e-ffba-4627-9b4e-3044c6c6896f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:37.443191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 551/1111] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:37.448325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:37.448818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:37.448912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:37.465869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:37.467091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:37.531737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 561/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:37.554726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fc91e-ffba-4627-9b4e-3044c6c6896f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:37.652798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fc91e-ffba-4627-9b4e-3044c6c6896f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:37.685499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:37.686324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:37.686602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:37.686872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] PATH_INFO: `/v3/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:37.688317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] RBAC: Authorizing `identity:get_role(role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:37.689711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:37.691050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 552/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:37 2022] GET /identity/v3/roles/2c535dd04ec64d44949ee9193f6d696e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:37.697051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:37.785010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:37.793422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd287ad-f92c-4823-a650-d0d09a63145e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:37.826914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:37.827982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:37.828658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:37.828952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] PATH_INFO: `/v3/users/01078c0986364120a555282fd8200c8c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:37.842546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] RBAC: Authorizing `identity:get_user(user_id=01078c0986364120a555282fd8200c8c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:37.843973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:37.845628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 562/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:37 2022] GET /identity/v3/users/01078c0986364120a555282fd8200c8c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:37.851158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:37.909028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:37.948955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30307b-2860-4476-a75e-c55e43f00f6e None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:37.982425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:37.983051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:37.983334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:37.983604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:38.049355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:38.050810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:38.107450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:38.154580 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 553/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:38.160600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:38.259379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a372f1-e816-465e-a74b-c043995c74b3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:38.291736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:38.292594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:38.292882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:38.293157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:38.294318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:38.295763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8b05d-a598-4b14-845a-9ea3ca715435 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:38.297468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 563/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:38 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:38.303430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:38.398054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e369a-9589-4d76-b848-af125a76e443 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:38.431133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:38.431760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:38.432043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:38.432310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:38.433152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3, user.id=01078c0986364120a555282fd8200c8c) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:38.433459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:38.434850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69338fc9-8c0b-478e-a5bd-4bd96d76a691 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:38.440743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 554/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:38 2022] GET /identity/v3/role_assignments?user.id=01078c0986364120a555282fd8200c8c&scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:40.613837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b5f69-9756-4fcf-bb94-d0d35db2c8c5 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:40.614395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b5f69-9756-4fcf-bb94-d0d35db2c8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:40.614395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b5f69-9756-4fcf-bb94-d0d35db2c8c5 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:40.615743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 564/1118] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:40.620934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:40.621334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:40.621495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:40.638593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:40.639771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:40.657400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:40.705380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 555/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 09:09:40.728420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b5f69-9756-4fcf-bb94-d0d35db2c8c5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:40.746924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:40.827550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b5f69-9756-4fcf-bb94-d0d35db2c8c5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:40.860244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:40.861143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:40.861428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:40.861703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] PATH_INFO: `/v3/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:40.863171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] RBAC: Authorizing `identity:get_role(role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:40.864592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:40.865906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 565/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:40 2022] GET /identity/v3/roles/2c535dd04ec64d44949ee9193f6d696e => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:40.871531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:40.968201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a52606-1c4b-4865-8da6-5f377a7b0af9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:41.000747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:41.001591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:41.001866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:41.002132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] PATH_INFO: `/v3/users/01078c0986364120a555282fd8200c8c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:41.003745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] RBAC: Authorizing `identity:get_user(user_id=01078c0986364120a555282fd8200c8c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:41.005391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:41.007069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 556/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:40 2022] GET /identity/v3/users/01078c0986364120a555282fd8200c8c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:41.012890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:41.095871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:41.110017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5abe-b72a-4606-8bf7-9823dc422e29 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:41.143734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:41.144523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:41.144834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:41.145116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:41.146277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:41.147700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0da442-d3fb-4c1d-b1c9-1ef419056c52 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:41.149399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 566/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:41 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:41.155168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:41.178634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:41.252115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db70e2-8e04-48ee-9324-87237dc67a48 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:41.285234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:41.286099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:41.286379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:41.286651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3/users/01078c0986364120a555282fd8200c8c/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:41.288855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] RBAC: Authorizing `identity:create_grant(project_id=84caeb95e5f64d12be326bc4c25d7cd3, user_id=01078c0986364120a555282fd8200c8c, role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:41.290265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25159-0af7-417a-81fb-ae6629aa6626 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:41.374428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 557/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:09:41 2022] PUT /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3/users/01078c0986364120a555282fd8200c8c/roles/2c535dd04ec64d44949ee9193f6d696e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:09:43.545770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db41ce-a4e9-42d2-8507-1a87e4ef29f1 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:43.546325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db41ce-a4e9-42d2-8507-1a87e4ef29f1 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:43.546325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db41ce-a4e9-42d2-8507-1a87e4ef29f1 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:43.547689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 567/1124] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:43.553038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:43.553477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:43.553571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:43.570867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:43.572018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:43.637226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 558/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:43.660426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db41ce-a4e9-42d2-8507-1a87e4ef29f1 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:43.757967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db41ce-a4e9-42d2-8507-1a87e4ef29f1 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:43.789792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:43.790625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:43.790903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:43.791172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] PATH_INFO: `/v3/roles/2c535dd04ec64d44949ee9193f6d696e` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:43.792683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] RBAC: Authorizing `identity:get_role(role_id=2c535dd04ec64d44949ee9193f6d696e)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:43.794036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:43.795964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 568/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:43 2022] GET /identity/v3/roles/2c535dd04ec64d44949ee9193f6d696e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:43.801432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:43.815508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:43.895569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16a96-a77c-4f80-8e98-1e24cf692eb5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:43.928983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:43.929829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:43.930104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:43.930374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] PATH_INFO: `/v3/users/01078c0986364120a555282fd8200c8c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:43.931531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] RBAC: Authorizing `identity:get_user(user_id=01078c0986364120a555282fd8200c8c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:43.932951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:43.934533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 559/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:43 2022] GET /identity/v3/users/01078c0986364120a555282fd8200c8c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:43.940082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:43.963983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:44.037934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab956be-05ce-4735-a590-ef541186a01f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:44.070718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:44.071296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:44.071579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:44.071855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:44.136460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:44.137941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:44.160537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:44.232883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 569/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:44.238678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:44.318106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:44.336833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9238-af9f-4efa-9664-5ae5204945ed None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:44.369763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:44.370596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:44.370872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:44.371139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] PATH_INFO: `/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:44.372294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] RBAC: Authorizing `identity:get_project(project_id=84caeb95e5f64d12be326bc4c25d7cd3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:44.373735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fec48-7dc2-4616-aafb-eed2abc8f898 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:44.375392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 560/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:44 2022] GET /identity/v3/projects/84caeb95e5f64d12be326bc4c25d7cd3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:44.381394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:44.478008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a993c-d414-4c55-adbd-1e7f0632a30a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:44.499552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:44.510982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:44.512553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:44.512553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:44.512553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:44.513519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] RBAC: Adding query filter params (role.id=2c535dd04ec64d44949ee9193f6d696e, scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3, user.id=01078c0986364120a555282fd8200c8c) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:44.513826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:44.515227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedac0f2-01d1-4568-8110-186ba1f76af9 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:44.520849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 570/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:44 2022] GET /identity/v3/role_assignments?user.id=01078c0986364120a555282fd8200c8c&scope.project.id=84caeb95e5f64d12be326bc4c25d7cd3&role.id=2c535dd04ec64d44949ee9193f6d696e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:09:46.740848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459db76-d11c-44d5-8284-945e3745c47b None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:46.741384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459db76-d11c-44d5-8284-945e3745c47b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:46.741384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459db76-d11c-44d5-8284-945e3745c47b None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:46.742771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 561/1131] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:46.748606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:46.749033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:46.749161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:46.765653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:46.766841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:46.830848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 571/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:46.853915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459db76-d11c-44d5-8284-945e3745c47b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:46.953589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459db76-d11c-44d5-8284-945e3745c47b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:46.986904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:46.987678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:46.987960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:46.988230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:46.989439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:46.991856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a358c-a25d-4485-95ca-3526133eb84b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:46.993442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 562/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:46.999231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:47.098268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72706-20cd-463f-8145-7af36b6958a8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:47.114497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:47.131308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:47.131979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:47.132262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:47.132540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:47.133274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:47.134959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95086f87-7ed3-4d4f-ba68-b95eb35f4a13 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:47.209150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 572/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:09:46 2022] POST /identity/v3/projects => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:49.437893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b3a08-24d9-4a65-97fa-0f8923cd7d77 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:49.438433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b3a08-24d9-4a65-97fa-0f8923cd7d77 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:49.438433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b3a08-24d9-4a65-97fa-0f8923cd7d77 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:49.439811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 563/1135] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:49.445059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:49.445493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:49.445627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:49.462605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:49.463789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:49.528433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 573/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:49.552698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b3a08-24d9-4a65-97fa-0f8923cd7d77 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:49.570884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:49.653151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b3a08-24d9-4a65-97fa-0f8923cd7d77 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:49.686907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:49.687674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:49.688013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:49.688233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:49.689424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:49.690833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e100cff-06b3-4064-8546-dfedf77e2d82 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:49.692412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 564/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:09:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:49.697970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:49.790165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:49.794408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453e2e2-e651-448b-b337-61b770bc1617 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:49.827522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:49.828248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:49.828524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:49.828833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:49.829537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:49.830980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8042760-149a-4e98-863b-d1ee7fb3e5de None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:49.894974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 574/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:09:49 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:09:52.145836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baae04d-f5a3-4500-9b27-f0635b0a247c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.146374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baae04d-f5a3-4500-9b27-f0635b0a247c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.146374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baae04d-f5a3-4500-9b27-f0635b0a247c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.147739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 565/1139] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:52.152951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.153370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.153511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.171016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:52.172199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:52.236856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 575/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:52.260631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baae04d-f5a3-4500-9b27-f0635b0a247c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:52.342254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:52.358535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baae04d-f5a3-4500-9b27-f0635b0a247c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:52.391162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:52.391932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.392211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.392481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.402390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:52.410292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:52.411414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:52.412446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 566/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:52.417239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:52.498048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:52.513563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c162ff-0231-4f96-8bd8-cd1cb6de1a7f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:52.546721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:52.547362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.547639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.547909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.548666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:52.548973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:52.550332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:52.558395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 576/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:52.563600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:52.661639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c071eb-8ab8-4890-8367-c666fc152fe5 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:52.693331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:52.694167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.694447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.694714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] PATH_INFO: `/v3/users/bf85f6fda9ae4825938b9bbd2ee22119` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.707867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] RBAC: Authorizing `identity:get_user(user_id=bf85f6fda9ae4825938b9bbd2ee22119)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:52.709310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:52.710912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 567/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:52 2022] GET /identity/v3/users/bf85f6fda9ae4825938b9bbd2ee22119 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:52.717192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:52.812383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedf0bc-dbbe-43be-b0eb-9b8d5a8a6e17 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:52.844855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:52.845481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:52.845761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:52.846038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:52.860440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:52.911142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:52.912568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:53.006867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 577/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:53.013411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:53.016281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:53.110412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4b6bc-6968-40f0-a652-fd8daf71a601 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:53.143010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:53.143843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:53.144123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:53.144396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] PATH_INFO: `/v3/projects/d9643d53bae24f128e5f0689942a4f32` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:53.145582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] RBAC: Authorizing `identity:get_project(project_id=d9643d53bae24f128e5f0689942a4f32)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:53.146982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfbb01-45fb-42c0-a13f-8204ca789e56 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:53.148666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 568/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:53 2022] GET /identity/v3/projects/d9643d53bae24f128e5f0689942a4f32 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:53.155131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:53.200071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:53.251218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bdb15-23ef-40dd-85be-3d7c74a4c12e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:53.283853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:53.284489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:53.284802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:53.285076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:53.285891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d9643d53bae24f128e5f0689942a4f32, user.id=bf85f6fda9ae4825938b9bbd2ee22119) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:53.286198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:53.287605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc216f7f-15c3-4190-87d2-c6241e9b6057 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:53.292894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 578/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:53 2022] GET /identity/v3/role_assignments?user.id=bf85f6fda9ae4825938b9bbd2ee22119&scope.project.id=d9643d53bae24f128e5f0689942a4f32&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:55.461286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341148ef-8a22-4c6a-b95c-79f0f8c302d4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:55.461811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341148ef-8a22-4c6a-b95c-79f0f8c302d4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:55.461905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341148ef-8a22-4c6a-b95c-79f0f8c302d4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:55.463221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 569/1147] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:55.469035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:55.469435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:55.469588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:55.486430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:55.487600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:55.552515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 579/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:55.575373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341148ef-8a22-4c6a-b95c-79f0f8c302d4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:55.645421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:55.677353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341148ef-8a22-4c6a-b95c-79f0f8c302d4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:55.710655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:55.711475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:55.711710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:55.711984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:55.722047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:55.729833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:55.730927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:09:55.732515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 570/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:55.737433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:55.777615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:55.833146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab3b8f-f535-47f8-85d6-5549cff16fc5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:55.865749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:55.866401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:55.866683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:55.866955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:55.867689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:55.867993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:55.869371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:55.877816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 580/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:55.883866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:55.990112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd549-801a-43b3-b191-c3bfa5849806 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:56.017588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:56.023271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:56.024132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:56.024458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:56.024769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] PATH_INFO: `/v3/users/bf85f6fda9ae4825938b9bbd2ee22119` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:56.025960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] RBAC: Authorizing `identity:get_user(user_id=bf85f6fda9ae4825938b9bbd2ee22119)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:56.027377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:56.029566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 571/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:55 2022] GET /identity/v3/users/bf85f6fda9ae4825938b9bbd2ee22119 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:56.035170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:56.130683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99aa0c-1ec7-4271-a9a9-0b3ce0982b42 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:56.132164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:56.163669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:56.164525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:56.165007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:56.165307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] PATH_INFO: `/v3/projects/d9643d53bae24f128e5f0689942a4f32` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:56.166834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] RBAC: Authorizing `identity:get_project(project_id=d9643d53bae24f128e5f0689942a4f32)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:56.168270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f32f5-6362-4fa3-9dae-66eece068bfe None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:56.170595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 581/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:56 2022] GET /identity/v3/projects/d9643d53bae24f128e5f0689942a4f32 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:56.176338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:56.274113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de05a6-9576-4b4e-84a3-504b30d4d9e0 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:56.307185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:56.308054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:56.308338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:56.308636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] PATH_INFO: `/v3/projects/d9643d53bae24f128e5f0689942a4f32/users/bf85f6fda9ae4825938b9bbd2ee22119/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:56.310607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9643d53bae24f128e5f0689942a4f32, user_id=bf85f6fda9ae4825938b9bbd2ee22119, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:56.312022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9cdb-2b3c-4afc-a476-510ae48d061d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:56.409389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 572/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:09:56 2022] PUT /identity/v3/projects/d9643d53bae24f128e5f0689942a4f32/users/bf85f6fda9ae4825938b9bbd2ee22119/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:09:58.617739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2c9f-5e83-4dc1-86f8-ef645ddd8ad8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:58.618338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2c9f-5e83-4dc1-86f8-ef645ddd8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:58.618338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2c9f-5e83-4dc1-86f8-ef645ddd8ad8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:58.619658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 582/1154] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:09:58.625624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:58.626030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:58.626183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:58.643115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:09:58.644290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:58.697265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:58.710225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 573/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:09:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:09:58.734376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd2c9f-5e83-4dc1-86f8-ef645ddd8ad8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:58.833785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd2c9f-5e83-4dc1-86f8-ef645ddd8ad8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:58.866311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:58.867085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:58.867369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:58.867644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:58.878630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:09:58.886418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:09:58.887692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:09:58.888983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 583/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:09:58.893550 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:58.988708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27127a6c-0f11-4efd-a445-48135b39a92d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.021081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.021734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.022010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.022278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.023007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:59.023307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.024688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.033114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 574/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:59.038819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:59.068703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:59.137692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096418d-faf9-4d27-8894-d62967f58281 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.170802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.171679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.171925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.172195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] PATH_INFO: `/v3/users/bf85f6fda9ae4825938b9bbd2ee22119` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.173387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] RBAC: Authorizing `identity:get_user(user_id=bf85f6fda9ae4825938b9bbd2ee22119)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.179096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.179096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 584/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:09:59 2022] GET /identity/v3/users/bf85f6fda9ae4825938b9bbd2ee22119 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:59.182296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:59.232230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:59.280531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd73a84-6e15-443e-8666-2334011709f7 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.313954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.314587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.314873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.315149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.381259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.382718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.430819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:59.481097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 575/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:09:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:09:59.486796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:59.579692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:59.583443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78703c5-4118-4223-9b6d-43537d48b77b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.616192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.617056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.617336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.617608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] PATH_INFO: `/v3/projects/d9643d53bae24f128e5f0689942a4f32` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.618762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] RBAC: Authorizing `identity:get_project(project_id=d9643d53bae24f128e5f0689942a4f32)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.620185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f043f5-1ba8-4646-84e0-bc5097198dec None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.621886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 585/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:09:59 2022] GET /identity/v3/projects/d9643d53bae24f128e5f0689942a4f32 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:09:59.627911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:59.724159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e2564-d4e3-40bf-a6e3-5e8d82a2897b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.756975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.757609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.757894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.758169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.759256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d9643d53bae24f128e5f0689942a4f32, user.id=bf85f6fda9ae4825938b9bbd2ee22119) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:09:59.759567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.761011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7b55a-cf6a-460e-aeed-ff11c26e8169 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.767251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 576/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:09:59 2022] GET /identity/v3/role_assignments?user.id=bf85f6fda9ae4825938b9bbd2ee22119&scope.project.id=d9643d53bae24f128e5f0689942a4f32&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:02.015897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb46ec-5548-43b3-9a35-378428cc1f7c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:02.016324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb46ec-5548-43b3-9a35-378428cc1f7c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:02.016436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb46ec-5548-43b3-9a35-378428cc1f7c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:02.017842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 586/1162] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:02.023123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:02.023554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:02.023680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:02.040627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:02.041807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:02.051506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:02.108058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 577/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:02.130178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb46ec-5548-43b3-9a35-378428cc1f7c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:02.229127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb46ec-5548-43b3-9a35-378428cc1f7c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:02.262077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:02.262676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:02.262963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:02.263241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:02.295676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:02.327532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:02.329316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:02.424106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 587/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:10:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:10:02.429994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:02.525861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a6b9b-a90a-4893-b2ed-91cb43d70b19 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:02.558161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:02.558938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:02.559215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:02.559486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:02.565979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:02.567384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-9d845165-7b7c-484c-8624-852aaacfa7df None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:10:02.581461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:10:02.582675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:10:02.583838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 578/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:10:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:10:02.588354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:02.651453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:02.684696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0f902-d9a4-47fd-a56e-70012ed71721 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:02.718408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:02.719055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:02.719338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:02.719612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:02.720326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:02.720663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:02.722032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd2d39-198a-4e56-b635-770b6a8b1a46 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:02.729009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 588/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Jun 7 09:10:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:04.920693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf32270-1c44-4e4e-aa93-ae76eba7a421 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:04.921241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf32270-1c44-4e4e-aa93-ae76eba7a421 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:04.921241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf32270-1c44-4e4e-aa93-ae76eba7a421 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:04.922617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 579/1167] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:04.927915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10f7b-abf6-40ca-9e74-f398515b48ec None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:04.928284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10f7b-abf6-40ca-9e74-f398515b48ec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:04.928478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10f7b-abf6-40ca-9e74-f398515b48ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:04.945436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-45c10f7b-abf6-40ca-9e74-f398515b48ec None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:04.946609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-45c10f7b-abf6-40ca-9e74-f398515b48ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:05.011151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 589/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:05.033894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf32270-1c44-4e4e-aa93-ae76eba7a421 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:05.038374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:05.135017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf32270-1c44-4e4e-aa93-ae76eba7a421 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:05.167828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:05.168522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:05.168840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:05.169121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:05.169867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:05.171278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df412baf-706c-4945-89a2-58f0da2e37a7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:05.261515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 580/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Tue Jun 7 09:10:05 2022] POST /identity/v3/domains => generated 267 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:10:07.479054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690d4d-14af-44b1-8509-7fcfc37b8aed None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:07.479590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690d4d-14af-44b1-8509-7fcfc37b8aed None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:07.479590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690d4d-14af-44b1-8509-7fcfc37b8aed None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:07.481591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 590/1170] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:07.486752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:07.487193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:07.487292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:07.503766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:07.504963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:07.561983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:07.569790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 581/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:07.594129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86690d4d-14af-44b1-8509-7fcfc37b8aed None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:07.654308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:07.692948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86690d4d-14af-44b1-8509-7fcfc37b8aed None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:07.725599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:07.726436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:07.726715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:07.726986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] PATH_INFO: `/v3/domains/9959133529f041019371aaaaec2486cd` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:07.737058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9959133529f041019371aaaaec2486cd)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:07.738430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4c87-898e-40a9-9ef4-18e30526c3b8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:07.740019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 591/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Jun 7 09:10:07 2022] GET /identity/v3/domains/9959133529f041019371aaaaec2486cd => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:07.745820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:07.841991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fb56c-ae09-4392-aee7-4de7197c2681 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:07.874513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:07.875151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:07.875431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:07.875705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:07.876413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:07.877891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4610500-2066-4cd3-8dff-ebfb1989be40 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:07.987449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 582/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:10:07 2022] POST /identity/v3/projects => generated 361 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:10:10.218966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f989-ffb0-487f-a026-303395309e32 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:10.219518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f989-ffb0-487f-a026-303395309e32 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:10.219518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f989-ffb0-487f-a026-303395309e32 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:10.220931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 592/1174] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:10.226087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:10.226479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:10.226649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:10.242995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:10.244166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:10.285696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:10.309481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 583/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:10.334229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604f989-ffb0-487f-a026-303395309e32 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:10.434112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604f989-ffb0-487f-a026-303395309e32 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:10.453528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:10.466920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:10.467766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:10.468051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:10.468327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] PATH_INFO: `/v3/domains/9959133529f041019371aaaaec2486cd` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:10.469532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9959133529f041019371aaaaec2486cd)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:10.471049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332aa4-9d22-4dc8-8a65-0aa945e44f67 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:10.472665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 593/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Jun 7 09:10:10 2022] GET /identity/v3/domains/9959133529f041019371aaaaec2486cd => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:10.478412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:10.575126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daae19-4042-4f9b-9f7e-e85486dd9548 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:10.608818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:10.609471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:10.609755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:10.610029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:10.610732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:10.612137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc3d1c-53dd-4d3e-9d20-adb3def72801 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:10.704816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 584/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:10:10 2022] POST /identity/v3/users => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:10:12.933081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1855-e589-481c-8139-e2c5796a4a88 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:12.933615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1855-e589-481c-8139-e2c5796a4a88 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:12.933615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1855-e589-481c-8139-e2c5796a4a88 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:12.935005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 594/1178] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:12.940080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:12.940502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:12.940653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:12.957497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:12.958679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:12.984240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:13.024702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 585/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:13.049477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a1855-e589-481c-8139-e2c5796a4a88 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:13.147899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a1855-e589-481c-8139-e2c5796a4a88 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:13.180292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:13.181101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:13.181388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:13.181662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:13.192019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:10:13.201913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:10:13.203003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:10:13.203895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 595/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:10:13.208978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:13.304359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319e887-9829-4d77-bb6c-753a77a00f22 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:13.337591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:13.338233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:13.338515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:13.338789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:13.339524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:13.339859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:13.341239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:13.349115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 586/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:13.354908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:13.403027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:13.452079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787edee-ed1e-46a2-a0f5-013e5cbf4c05 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:13.484151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:13.485025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:13.485311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:13.485585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] PATH_INFO: `/v3/users/f7d278f23cb3428fa4e736a6c7ad1042` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:13.498601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] RBAC: Authorizing `identity:get_user(user_id=f7d278f23cb3428fa4e736a6c7ad1042)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:13.500035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:13.501671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 596/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/users/f7d278f23cb3428fa4e736a6c7ad1042 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:13.507186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:13.532457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:13.609371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2d47-c3bf-4d9f-a1fc-60247e924d4f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:13.645408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:13.646086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:13.646373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:13.646648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:13.711566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:13.712966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:13.736141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:13.808518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 587/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:10:13.814323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:13.911518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998011a7-42ff-4977-885e-c1982d08267e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:13.918570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:13.943911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:13.944785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:13.945069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:13.945342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] PATH_INFO: `/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:13.946517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] RBAC: Authorizing `identity:get_project(project_id=71fc0f1f636a4a7ca841f8a0a532be70)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:13.947936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902f9b-1985-4dda-ba31-188f3980b596 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:13.950253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 597/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:13.956341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:14.055149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a2c8-d2c9-4420-b663-1c5fba954688 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:14.078693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:14.089064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:14.089726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:14.090014 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:14.090289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:14.094628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=71fc0f1f636a4a7ca841f8a0a532be70, user.id=f7d278f23cb3428fa4e736a6c7ad1042) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:14.094628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:14.094628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cbaa5-e168-49cf-a240-6321de30d209 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:14.098236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 588/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:10:13 2022] GET /identity/v3/role_assignments?user.id=f7d278f23cb3428fa4e736a6c7ad1042&scope.project.id=71fc0f1f636a4a7ca841f8a0a532be70&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:16.276629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d561465-65c1-4ada-8eb1-d4c5cc1c4abb None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.277131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d561465-65c1-4ada-8eb1-d4c5cc1c4abb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.277131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d561465-65c1-4ada-8eb1-d4c5cc1c4abb None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.278977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 598/1186] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:16.284067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.284610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.284720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.301655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:16.302830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:16.368733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 589/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:16.391142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d561465-65c1-4ada-8eb1-d4c5cc1c4abb None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:16.488555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d561465-65c1-4ada-8eb1-d4c5cc1c4abb None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:16.520561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:16.521356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.521639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.521908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.532159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:10:16.539684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:10:16.540818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:10:16.541708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 599/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:10:16.546577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:16.646351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e46fe7-b3ce-44d7-92db-0de990f35539 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:16.663694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:16.680463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:16.681152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.681439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.681792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.682509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:16.682817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:16.684879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:16.693237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 590/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:16.698800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:16.732322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:16.798257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bf65a-e420-4989-b6d1-d6013909cab3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:16.830998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:16.831844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.832128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.832403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] PATH_INFO: `/v3/users/f7d278f23cb3428fa4e736a6c7ad1042` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.833597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] RBAC: Authorizing `identity:get_user(user_id=f7d278f23cb3428fa4e736a6c7ad1042)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:16.835028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:16.836731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 600/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:10:16 2022] GET /identity/v3/users/f7d278f23cb3428fa4e736a6c7ad1042 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:16.842405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:16.941419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ce2ce-859a-406a-b380-004874824de6 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:16.974006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:16.974856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:16.975272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:16.975552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] PATH_INFO: `/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:16.976751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] RBAC: Authorizing `identity:get_project(project_id=71fc0f1f636a4a7ca841f8a0a532be70)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:16.978168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9699eab-135a-4aca-9894-e3e98c43bb6e None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:16.979832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 591/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:10:16 2022] GET /identity/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:16.986059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:17.082874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d95d-c496-4ff5-b2b6-b901d4580a24 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:17.084306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:17.115804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:17.116710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:17.117224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:17.117723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] PATH_INFO: `/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70/users/f7d278f23cb3428fa4e736a6c7ad1042/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:17.119974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] RBAC: Authorizing `identity:create_grant(project_id=71fc0f1f636a4a7ca841f8a0a532be70, user_id=f7d278f23cb3428fa4e736a6c7ad1042, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:17.121477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd44e61-c70b-4b5e-8e0c-e4112261f202 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:17.179821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 601/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:10:16 2022] PUT /identity/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70/users/f7d278f23cb3428fa4e736a6c7ad1042/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:10:19.374754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039b66-15d6-489c-bba0-f233dd8f91f7 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:19.375259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039b66-15d6-489c-bba0-f233dd8f91f7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:19.375259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039b66-15d6-489c-bba0-f233dd8f91f7 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:19.376729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 592/1193] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:10:19.381789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:19.382215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:19.382323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:19.399454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:10:19.400635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:19.463481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 602/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:10:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:10:19.487035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039b66-15d6-489c-bba0-f233dd8f91f7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:19.537417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:19.587943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039b66-15d6-489c-bba0-f233dd8f91f7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:19.620340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:19.621719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:19.622004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:19.622276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:19.632159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:10:19.640015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:10:19.641130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:10:19.642032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 593/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:10:19.647130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:19.734945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:19.743472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c7b15-8fe2-414c-9038-577a8eb359e2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:19.777927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:19.778573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:19.778857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:19.779133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:19.779869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:19.780174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:19.781581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:19.789447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 603/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:19.795490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:19.891350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c105-38ca-46fb-95f3-c71040a1ea7b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:19.899777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:19.923615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:19.924455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:19.924766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:19.925039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] PATH_INFO: `/v3/users/f7d278f23cb3428fa4e736a6c7ad1042` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:19.926205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] RBAC: Authorizing `identity:get_user(user_id=f7d278f23cb3428fa4e736a6c7ad1042)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:19.927618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:19.929235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 594/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Jun 7 09:10:19 2022] GET /identity/v3/users/f7d278f23cb3428fa4e736a6c7ad1042 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:19.935470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:20.033089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bedd-b4a8-4074-956d-715c33f93542 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:20.065929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:20.066567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:20.066854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:20.067133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:20.090851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:20.133231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:20.134670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:20.228450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 604/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:10:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:10:20.234305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:20.333198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d381f-0388-41ae-81b3-1a2ea4b9845b None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:20.365868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:20.366700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:20.366982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:20.367262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] PATH_INFO: `/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:20.368449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] RBAC: Authorizing `identity:get_project(project_id=71fc0f1f636a4a7ca841f8a0a532be70)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:20.369903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8fa21-3166-40b9-9165-e97bf862144c None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:20.372153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 595/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Jun 7 09:10:20 2022] GET /identity/v3/projects/71fc0f1f636a4a7ca841f8a0a532be70 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:20.378035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:20.431169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:20.473659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f863bc-1920-49a3-99ed-7deae55e196e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:20.506525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:20.507104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:20.507385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:20.507659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:20.508770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=71fc0f1f636a4a7ca841f8a0a532be70, user.id=f7d278f23cb3428fa4e736a6c7ad1042) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:10:20.509078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:20.510502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea343-9420-4e82-b146-cad7a61a6b1a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:20.516637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 605/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:10:20 2022] GET /identity/v3/role_assignments?user.id=f7d278f23cb3428fa4e736a6c7ad1042&scope.project.id=71fc0f1f636a4a7ca841f8a0a532be70&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:10:30.361896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=78 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:30.514047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=41 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:41.007479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475907d-72af-4bbe-be0d-a9591a3bc1ad None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.007967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475907d-72af-4bbe-be0d-a9591a3bc1ad None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.008070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475907d-72af-4bbe-be0d-a9591a3bc1ad None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.009431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 596/1201] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:11:41.015241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.015675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.015798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.032421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:41.033646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:41.098822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 606/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:11:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:11:41.123764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475907d-72af-4bbe-be0d-a9591a3bc1ad None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.224147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475907d-72af-4bbe-be0d-a9591a3bc1ad None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.259251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.259251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.259251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.259251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.269456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.270916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:41.287906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:41.289126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:41.290321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:41.290321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:41.290321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:41.290321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:41.290321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 597/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:41.295153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.392952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b38d-ace1-47bb-8834-1913c551f206 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.420851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:41.426527 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.427178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.427464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.427742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.428458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:41.428798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.430222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e96c86-e78b-442a-8b4d-f6d0156c3b7e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.441656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 607/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:41.447363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.499680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:41.544187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091e54-c5d2-4d7e-9f75-29dd3105a2c1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.576916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.577568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.577846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.578115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.578815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.580324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57fe4-65e1-40c4-92e3-12951ee89336 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.664526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 598/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Jun 7 09:11:41 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:43.901666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446baa4f-78c0-4601-98de-b8dd9fc41012 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.902242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446baa4f-78c0-4601-98de-b8dd9fc41012 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.902242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446baa4f-78c0-4601-98de-b8dd9fc41012 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.904138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 608/1206] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:11:43.909326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.909726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.909891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.926704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:43.927948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:43.993803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 599/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:11:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:11:44.018768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446baa4f-78c0-4601-98de-b8dd9fc41012 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.119701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446baa4f-78c0-4601-98de-b8dd9fc41012 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.154930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.155650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.155931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.156203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.166440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:11:44.174039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:44.175058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:44.175937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 609/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:44.181219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.220117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:44.277480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9675d9-2e41-494a-a37f-7ad7fdc40ce8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.310849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.311496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.311777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.312049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.312813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:44.313116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:44.314457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:44.322318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 600/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:44.328608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.378124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:44.425285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ce9a-331e-4e8e-988b-0f0b9a0c26d2 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.457805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.458580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.458862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.459134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.468895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:44.470328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:44.488327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:44.489446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:44.490390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:44.490390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:44.490390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:44.490390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:44.490390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 610/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:44.495181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.592257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:44.595270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5cab-19b2-413e-b876-fd70212d6624 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.628485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.629247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.629527 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.629806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.630524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:44.630827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:44.632444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:44.644197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 601/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:44.650318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.751011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279878-64df-413b-bf29-076a78d27d82 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.752552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:44.784951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.785742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.786024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.786297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.793786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:44.795279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:11:44.810076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:44.814025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:44.815019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 611/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:44.817224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:44.915729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6899d-f262-4a82-8520-977071bcad1a None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:44.949539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:44.950196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.950480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.950770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.951585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:44.951895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:44.953351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:44.970726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 602/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:44.976394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.075187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284a09a-5979-4d4d-9fa9-42ee1780555f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.107989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.108636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.108922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.109193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.139183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:45.175032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.176468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.269160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 612/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:11:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:11:45.274692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.293671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:45.373346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284abc88-6c16-4ad1-843d-2b288826ad93 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.407881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.408686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.408975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.409250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.419640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.421121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:45.439147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:45.440238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:45.441352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:45.441352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:45.441352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:45.441352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:45.441352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 603/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:45.446172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.505242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:45.541669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7785d4c-9f08-46bc-8e31-88d9801b627a None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.573794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.575027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.575305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.575576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.576305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:45.576623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.580781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.588665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 613/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:45.594385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.663589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:45.693110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c47d6-ac58-49d0-abda-2843a2e589ed None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.725686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.726464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.726744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.727016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.733505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.736318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:11:45.749013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:45.750166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:45.751157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 604/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:45.756552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.852662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156e7e-d2b4-40cf-8e76-e8a2741b54a1 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.874815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:45.886010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.886657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.886961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.887235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.888010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:45.888318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.889796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a38bc-5032-43cb-b8bb-d49035272ad4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.901835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 614/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:45.907908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:46.010900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358d933-9867-42fc-a48e-ede359702cba None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:46.039063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:46.045447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:46.046172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:46.046454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:46.046855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:46.047688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=6459963216fe4a039289498f1fd8af54) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:46.048025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:46.049485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94136edf-3d1e-4061-b175-b7b036f99b46 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:46.054973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 605/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/role_assignments?user.id=6459963216fe4a039289498f1fd8af54&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:48.249753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24f06-3244-4c33-966f-55fc5e274d3a None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.250298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24f06-3244-4c33-966f-55fc5e274d3a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.250298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24f06-3244-4c33-966f-55fc5e274d3a None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.251729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 615/1220] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:11:48.262222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.262222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.262222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.275560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:48.276766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:48.343315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 606/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:11:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:11:48.365590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24f06-3244-4c33-966f-55fc5e274d3a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.464737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24f06-3244-4c33-966f-55fc5e274d3a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.498283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.499068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.499349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.499622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.509813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:11:48.521722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:48.522858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:48.524022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 616/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:48.525243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.622699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cdffb-93ab-49f6-8517-f2d68912b8d5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.647110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:48.657891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.663384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.669271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 607/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:48.675172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.718942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:48.774805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70047-6c76-48aa-9bcc-ff3fa48c4000 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.806909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.807690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.807973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.808250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.819059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.824020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:48.837619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:48.838679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:48.839815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:48.839815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:48.839815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:48.839815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:48.839815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 617/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:48.844560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.946653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e6e7-5fd5-43d2-b88f-ea84762b78c3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.980001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.981210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.981494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.981767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.982484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:48.982792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.984148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.995939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 608/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:49.002170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.094066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:49.100457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb3811-1d75-4eb3-af72-981da2059303 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.134108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.134900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.135184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.135461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.142764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.144251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:11:49.158288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:49.159432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:49.160883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 618/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:49.165448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.212359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:49.264523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c9a1-b328-4a1a-ade8-5d44c8f85d1d None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.301648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.302309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.302653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.302926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.303746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:49.304050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.305481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.322542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 609/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:49.328893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.426227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d3c3e-d91a-4b79-b162-be8eb6c5bf59 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.459212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.459981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.460262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.460535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.471025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.472480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:49.489339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:49.490414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:49.491804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:49.491804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:49.491804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:49.491804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:49.491804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 619/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:49.496721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.596285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88966e-5375-4bf9-a9d8-a63669968f41 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.598565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:49.629481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.630135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.630411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.630684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.631647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:49.631951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.633321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.644333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 610/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:49.650031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.660380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:49.748411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acedd9-8e04-4a8f-afbe-da3f830f906d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.780357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.781161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.781447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.781723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.788308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.789781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:11:49.804093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:49.805219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:49.806495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 620/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:11:49.811718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:49.911119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523caca5-1a48-4609-bcbf-0061ff770257 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:49.945296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:49.945951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:49.946233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:49.946505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:49.947285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:49.947596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:49.949118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc84d2-9b7a-48fc-a532-99bca6af4a7a None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:49.961397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 611/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:49.974551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:50.033884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:50.067482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbbe46-0516-4119-9fe3-5fb27eab8745 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:50.102825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:50.103699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:50.103975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:50.104242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/6459963216fe4a039289498f1fd8af54/roles/76cbe6c8ed30411d8ca4a51d7300fc69` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:50.118292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=6459963216fe4a039289498f1fd8af54, role_id=76cbe6c8ed30411d8ca4a51d7300fc69)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:50.119720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0524-d90b-4871-9a8e-0a29f3b41451 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:50.224209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 621/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:11:49 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/6459963216fe4a039289498f1fd8af54/roles/76cbe6c8ed30411d8ca4a51d7300fc69 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:52.479354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da53985-9f99-4d5d-b44e-6de464fd1c1e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.479918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da53985-9f99-4d5d-b44e-6de464fd1c1e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.479918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da53985-9f99-4d5d-b44e-6de464fd1c1e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.481389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 612/1233] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:11:52.486725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.487143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.487282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.503564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:52.504794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:52.569882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 622/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:11:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:11:52.594038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da53985-9f99-4d5d-b44e-6de464fd1c1e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:52.619601 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:52.693730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da53985-9f99-4d5d-b44e-6de464fd1c1e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:52.725439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:52.726211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.726486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.726756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.741442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:11:52.744424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:52.745530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:52.746953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 613/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Jun 7 09:11:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:52.752481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:52.798934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:52.849941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda31f3-e6bd-45d1-835b-d9503e381e69 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:52.883671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:52.884331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.884640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.884919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.885657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] RBAC: Adding query filter params (name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:52.885985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:52.887489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:52.895701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 623/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Jun 7 09:11:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:52.901481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:52.986865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:53.002251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d312a-c1b5-4290-a7d7-7a637f752b51 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.036307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.037126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.037412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.037685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.048168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.049639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:53.066731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:53.067828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:53.068796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:53.068796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:53.068796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:53.068796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:53.068796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 614/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:53.073778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.172048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b044cd8-e9ac-46f6-b198-119b451c805e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.181131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:53.205312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.206026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.206350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.206628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.207350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:53.207709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.209126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.220824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 624/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:53.226376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.323885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30bee9-21da-4538-9bac-b13e493441b2 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.357740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.358580 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.358856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.359123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.361874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:53.366394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.367810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:11:53.382368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:53.383546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:53.384747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 615/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:53.389792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.487943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc1f9-68d2-4d2e-b8f2-415a7e6f342e None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.520721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.521395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.521685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.521958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.522792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:53.523100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.524537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.541854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 625/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:11:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:53.547574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.648529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa585ed-6655-4759-97b9-bd9280eea918 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.682217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.682848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.683119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.683920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.743282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:53.750453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.751923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.848371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 616/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:11:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:11:53.854437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.882957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:53.951888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a879-452b-4413-aef4-1774bb3818de None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.983871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.984685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.984970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.985248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.995052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.998863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:54.019360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:54.020556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:54.021970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:54.021970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:54.021970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:54.021970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:54.021970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 626/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:54.026219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.119545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.125788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b9721-7652-4db1-9a86-c7102657165f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.157990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.158640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.158920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.159194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.159913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:54.160227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.161601 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.172883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 617/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:54.178439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.256635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.276800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1623-6dfa-483a-baaa-2460e37e2393 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.309676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.310666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.311102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.311764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.318296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.319731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:11:54.333400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:54.334540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:54.335485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 627/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:54.340370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.438192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c955-dbf8-47b7-8f0e-9222b3c4fe44 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.470311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.470956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.471235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.471506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.472275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:54.472594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.474021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6efb4b-62ea-46f7-9088-73c77f958688 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.486428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 618/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:54.492431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.590930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a94bc-203b-4b35-be13-a6e4adc0b06d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.624626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e92ae-6734-4358-9949-2801913578ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.625277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e92ae-6734-4358-9949-2801913578ad None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.625563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e92ae-6734-4358-9949-2801913578ad None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.625844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e92ae-6734-4358-9949-2801913578ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.626671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e92ae-6734-4358-9949-2801913578ad None admin] RBAC: Adding query filter params (role.id=76cbe6c8ed30411d8ca4a51d7300fc69, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=6459963216fe4a039289498f1fd8af54) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:54.626988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e92ae-6734-4358-9949-2801913578ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.628467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e92ae-6734-4358-9949-2801913578ad None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.634798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 628/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/role_assignments?user.id=6459963216fe4a039289498f1fd8af54&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=76cbe6c8ed30411d8ca4a51d7300fc69 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:56.908351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4af8-9afa-4ce9-b3f4-22afc079dc10 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:56.908915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4af8-9afa-4ce9-b3f4-22afc079dc10 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:56.908915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4af8-9afa-4ce9-b3f4-22afc079dc10 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:56.910323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 619/1247] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:11:56.915649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:56.916082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:56.916211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:56.921826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:56.933709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:56.934879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:57.000682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 629/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:11:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:11:57.024951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf4af8-9afa-4ce9-b3f4-22afc079dc10 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.044045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:57.125657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf4af8-9afa-4ce9-b3f4-22afc079dc10 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.159923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.160698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.160985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.161261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.171182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:11:57.178842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:57.179981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:57.181037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 620/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:57.186211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.285489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfec103-dfc1-4a32-8341-c4675df8d043 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.313812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:57.318720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.319383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.319666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.319941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.320717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:57.321025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:57.322411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:57.330777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 630/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:57.336445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.414623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:57.436870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca25-cf6e-42fd-9970-935c3e18afb1 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.468836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.469605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.469913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.470184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.480880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:57.482304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:57.499811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:57.500917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:57.502049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:57.502049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:57.502049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:57.502049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:57.502049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 621/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:57.507010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.606609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36073f-a2c6-4f6d-8b23-93469cc78c81 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.639877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.640531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.640846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.641122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.641843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:57.642151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:57.643517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:57.655604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 631/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:57.664159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.760876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f125-1647-4cba-acb9-ab2ff878f538 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.787113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:57.795699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.796487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.796802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.797080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.804377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:57.805841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:11:57.820053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:57.821216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:11:57.822463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 622/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:11:57.827871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:57.881925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:57.924221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c4e34-cc45-4019-b2b0-4519375afa9d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:57.956856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:57.957523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:57.957801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:57.958071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:57.958929 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:57.959231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:57.960668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:57.977742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 632/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:57.983631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.084297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68a7b5-fd08-4f1f-988f-ee84695d7ca4 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:58.116625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:58.117263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:58.117540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:58.117809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:58.172357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:58.184469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:58.185923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:58.283586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 623/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:11:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:11:58.289362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.385847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd621fa-1c4f-4aea-b649-ae2b2b7a513d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:58.398917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:58.418962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:58.419743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:58.420024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:58.420299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:58.429960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:58.431394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:11:58.448707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:58.450243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:58.451481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:11:58.451481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:11:58.451481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:11:58.451481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:58.451481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 633/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:11:58.456076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.541367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:58.552842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e099cc-bdc7-4db6-a512-95ce19da8483 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:58.586922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:58.587586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:58.587864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:58.588145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:58.588894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:58.589204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:58.590595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:58.601701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 624/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:58.607862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.705309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ef717-1de5-40e6-ae96-01fad3367a1c None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:58.738882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:58.739655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:58.740028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:58.740900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:58.747327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:58.748791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:11:58.762545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:11:58.763607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:11:58.764715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 634/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:11:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:11:58.769487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.867894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c1a7-72e8-4bfa-b40e-950c77ed51eb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:58.901121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:58.901767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:58.902051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:58.902323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:58.903095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:58.903406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:58.904840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b4e2-db35-4a57-8a0e-a256840d98c7 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:58.916558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 625/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:58.923176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:58.948688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:59.022001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0d782-47c5-4bbd-9363-5388d4e8d29b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:59.055958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:59.056625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:59.056918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:59.057242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:59.058085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=6459963216fe4a039289498f1fd8af54) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:59.058593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:59.060923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f562-b721-4fdc-aad1-cdb366315dba None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:59.066340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 635/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:11:58 2022] GET /identity/v3/role_assignments?user.id=6459963216fe4a039289498f1fd8af54&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:01.258503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33ad41-b1d8-45ac-bef6-1986c0ec13cd None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.259066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33ad41-b1d8-45ac-bef6-1986c0ec13cd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.259066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33ad41-b1d8-45ac-bef6-1986c0ec13cd None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.260348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 626/1261] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:01.265638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.266060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.266180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.282949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:01.284132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:01.350098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 636/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:01.373312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33ad41-b1d8-45ac-bef6-1986c0ec13cd None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:01.384428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:01.473726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33ad41-b1d8-45ac-bef6-1986c0ec13cd None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:01.507623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:01.508411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.508785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.509050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.519394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:12:01.527770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:01.529006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:01.530123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 627/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:01.535126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:01.556359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:01.634162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d8e78-63b7-440c-a2f4-e9d036be3efd None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:01.668014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:01.668722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:01.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:01.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:01.679507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 637/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:01.685535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:01.751831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:01.785211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776f813-af12-4712-8e8a-70178d094f68 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:01.816738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:01.817696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.817879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.818145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.829246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:01.830731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:12:01.847609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:01.848743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:01.849863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:12:01.849863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:12:01.849863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:01.849863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:01.849863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 628/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:01.854530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:01.936840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:01.956773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f9249-e13d-4c79-91b5-41ea1d856e71 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:01.990483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:01.991142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.991427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.991703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.992423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:01.992761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:01.994128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.006621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 638/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:02.015534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.111995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8a3f-2a1c-4ab6-94bd-27a7a21e89bd None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.132047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:02.146271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.147057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.147341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.147622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.155011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.156445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:12:02.170488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:02.171681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:02.172699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 629/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 09:12:02.177436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.275583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2bc2-d27f-4433-97a1-444aa10b0ed3 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.308716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.309378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.309661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.309935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.310757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:02.311065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.312485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.320773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:02.330349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 639/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:02.336133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.436316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd2b3-b9de-4702-8e3d-081e43543b44 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.469097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.469875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.470159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.470435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.481451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.482946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:12:02.500279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:02.501410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:02.502384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:12:02.502384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:12:02.502384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:02.502384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:02.502384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 630/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:02.507475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.602742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea283c1-a0be-4267-8c03-a9c42c2e8bd6 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.636332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.637006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.637288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.637558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.638266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:02.638566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.639955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.650867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 640/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:02.656746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.689011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:02.757198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e1aa-9a51-4968-b5ec-fc5c7c6cec69 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.793296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.794072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.794355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.794621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.801096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.802544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:12:02.815771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:02.817707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:02.818837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 631/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:02.822848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.882287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:02.921905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d3b7-a9fe-492d-ab4c-36c8c942783b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.953381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.953989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.954256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.954527 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.955301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:02.955606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.957066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0814cb-3d0e-4a8c-aa73-fbd067ea89b2 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.969220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 641/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:02.976273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:03.067419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:03.076455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4054c-9053-4103-9761-0fb113cfcb01 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:03.109672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:03.110543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:03.110827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:03.111105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] PATH_INFO: `/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/6459963216fe4a039289498f1fd8af54/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:03.113079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] RBAC: Authorizing `identity:create_grant(project_id=d05821a6472b4ed48bf91eaae8471b52, user_id=6459963216fe4a039289498f1fd8af54, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:03.114510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b78f-bd8b-4722-beec-baf788ae8d3f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:03.224723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 632/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Jun 7 09:12:02 2022] PUT /identity/v3/projects/d05821a6472b4ed48bf91eaae8471b52/users/6459963216fe4a039289498f1fd8af54/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:05.406038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917eec4-b9c0-4c7c-8add-dc624bd0c4c8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.406595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917eec4-b9c0-4c7c-8add-dc624bd0c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.406595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917eec4-b9c0-4c7c-8add-dc624bd0c4c8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.407860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 642/1274] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:05.413030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.413432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.413593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.429986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:05.431150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:05.497193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 633/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:05.520466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917eec4-b9c0-4c7c-8add-dc624bd0c4c8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:05.619369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917eec4-b9c0-4c7c-8add-dc624bd0c4c8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:05.636564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:05.652938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:05.653714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.653991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.654259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.663839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 09:12:05.671360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:05.672373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:05.673423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 643/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Jun 7 09:12:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:05.678623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:05.765369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:05.782792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14b448-376b-4f71-9bb6-2301ad97a678 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:05.817432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:05.818097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.818381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.818659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.819409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:05.819718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:05.821127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:05.830015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 634/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Jun 7 09:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:05.835625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:05.932415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b4976-18fa-4c0d-8d68-b9ab2fad7da0 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:05.970208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:05.970986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.971263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.971531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.982420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:05.983897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:12:06.001539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:06.002530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:06.003580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:12:06.003580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:12:06.003580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:06.003580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:06.008208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 644/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:06.009437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.109223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3a4d-c09e-4e5f-ae93-41edc71d1dbb None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.142518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.143195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.143477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.143751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.144468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:06.144810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.146160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.157773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 635/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:06.163450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.195279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.268650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4776c-fce8-46a7-be5a-e06e58810668 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.301374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.302154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.302428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.302695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.309837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.311267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 09:12:06.325516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:06.326786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:06.327783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 645/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:06.332600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.336458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.430497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f343d0-1609-48b3-b916-1315c98ffc94 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.465460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.466056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.466337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.466611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.467468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:06.467777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.469240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.486335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 636/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:12:06.491984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.581431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.590863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b362a2-2ea6-49df-ba72-75f84f4dcd55 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.622897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.623533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.623816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.624090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.689405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.690847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.777029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.786761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 646/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 09:12:06.792627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.852928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.890551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399bf0-ee00-4ca5-836b-8f4e7415a0a3 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.924188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.924993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.925273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.925641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.935803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.937331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 09:12:06.955192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:06.956295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:06.957460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 09:12:06.957460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 09:12:06.957460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 09:12:06.957460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:06.957460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 637/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:06.962250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.061272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55b20-e10e-48cb-beb7-f9caf2f5cd36 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.094768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.095425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.095708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.095983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.096723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:07.097033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.098387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.109995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 647/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:07.115825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.212818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c5c47-83f8-4943-9f4d-1a14ad425a83 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.227304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:07.247125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.248927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.248927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.248927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.255090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.256830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 09:12:07.271431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:07.276054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: ERROR keystone.server.flask.application Jun 07 09:12:07.277742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 638/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Jun 7 09:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:07.279025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.328331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:07.378264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c12760-f90c-4bc2-b9d6-e19dd049df4d None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.411395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.412088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.412373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.412680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.413505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:07.413812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.415224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cb1b2-46af-4194-902c-6472f0df124c None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.427596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 648/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Jun 7 09:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:07.433639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.533028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4488-4d14-48eb-a2c4-14aaa2c60a23 None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.566980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.567624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.567905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.568179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.569039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] RBAC: Adding query filter params (role.id=8c0f69603d9b44b0b45e7b500a8f109a, scope.project.id=d05821a6472b4ed48bf91eaae8471b52, user.id=6459963216fe4a039289498f1fd8af54) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:07.569348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.570784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0ddb-a8a5-4d17-9988-9d95845f8be1 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.577111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 639/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Jun 7 09:12:07 2022] GET /identity/v3/role_assignments?user.id=6459963216fe4a039289498f1fd8af54&scope.project.id=d05821a6472b4ed48bf91eaae8471b52&role.id=8c0f69603d9b44b0b45e7b500a8f109a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:09.835477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7a5b-78ec-453b-bc54-c7d9a064ef32 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:09.836034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7a5b-78ec-453b-bc54-c7d9a064ef32 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:09.836139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7a5b-78ec-453b-bc54-c7d9a064ef32 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:09.837469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 649/1288] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:09.842922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:09.843300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:09.843476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:09.860569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:09.861782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:09.892435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:09.928676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 640/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:09.951774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc7a5b-78ec-453b-bc54-c7d9a064ef32 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:10.054352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc7a5b-78ec-453b-bc54-c7d9a064ef32 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:10.065581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:10.088772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:10.090080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:10.090363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:10.090625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:10.091279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:10.092668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:10.104718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:10.105963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:10.106976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 650/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:12:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:10.112074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:10.210846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8d2d-0665-4306-91ec-0a2bff5d6de0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:10.243812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:10.244467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:10.244836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:10.245115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:10.246409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:10.246718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:10.248074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b2a1-adf1-4db7-9abe-24c14325de31 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:10.252925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 641/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Jun 7 09:12:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:10.258866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:10.357994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3728f-dfd3-4bdc-aae1-230161c040e8 None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:10.390931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:10.391574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:10.391851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:10.392118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] PATH_INFO: `/v3/services` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:10.392847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:10.393152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:10.394505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93074cd-63fe-4c66-9db3-0dc9b3f1ae99 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:10.398807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 651/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Jun 7 09:12:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:12.575741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07160e5-05bc-45d9-9112-097366cc92e5 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:12.576273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07160e5-05bc-45d9-9112-097366cc92e5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:12.576273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07160e5-05bc-45d9-9112-097366cc92e5 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:12.577793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 642/1293] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:12.583260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f140-e02c-4ebd-9212-77684a7311da None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:12.583698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f140-e02c-4ebd-9212-77684a7311da None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:12.583851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f140-e02c-4ebd-9212-77684a7311da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:12.601019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5204f140-e02c-4ebd-9212-77684a7311da None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:12.602196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5204f140-e02c-4ebd-9212-77684a7311da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:12.631705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:12.668196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 652/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:12.691319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07160e5-05bc-45d9-9112-097366cc92e5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:12.724156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:12.794358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07160e5-05bc-45d9-9112-097366cc92e5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:12.828041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:12.828663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:12.828951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:12.829228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:12.829942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:12.831297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096462a-2d62-4211-adf3-fd27474562a5 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:12.910099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 643/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Jun 7 09:12:12 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:15.165669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7395ab-b7a6-441f-a77d-6a2b70c471ba None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.166259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7395ab-b7a6-441f-a77d-6a2b70c471ba None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.166259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7395ab-b7a6-441f-a77d-6a2b70c471ba None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.168130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 653/1296] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:15.173340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.173770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.173889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.191024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:15.196060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:15.265070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 644/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:15.287660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7395ab-b7a6-441f-a77d-6a2b70c471ba None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:15.385500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7395ab-b7a6-441f-a77d-6a2b70c471ba None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:15.418875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:15.419651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.419927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.420196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.420910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:15.422275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:15.434403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:15.437499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:15.438526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 654/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:12:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:15.441767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:15.460394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:15.540921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ac989-6749-401d-a415-dfdd019e7b8c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:15.576153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:15.576844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.577128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.577401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.578102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:15.578406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:15.579776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:15.584933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 645/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Jun 7 09:12:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:12:15.590903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:15.622616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:15.689523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d84f62-a943-4e49-bf9f-309baca0650f None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:15.723804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:15.724445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.724829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.725107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.725893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] RBAC: Adding query filter params (interface=public, service_id=bac3db1ca0a6403e840f1d6607de700e, region_id=RegionOne) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:15.726205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:15.727560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792ebfa-2235-4cd0-afcd-4051cb60e5f1 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:15.732473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 655/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Jun 7 09:12:15 2022] GET /identity/v3/endpoints?service_id=bac3db1ca0a6403e840f1d6607de700e&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:15.738099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:15.947052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:15.964372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe026f-41e3-49ea-b7ee-c57ec318610f None admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:15.997953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:15.998600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:15.998882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:15.999156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:15.999864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:16.001243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d0c47-ca4d-415c-a45f-7593387b1296 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:16.013235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 646/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Jun 7 09:12:15 2022] GET /identity/v3/services => generated 2191 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:18.249352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa22c-8323-4eb9-bf3c-b623cde687e3 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.249904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa22c-8323-4eb9-bf3c-b623cde687e3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.249904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa22c-8323-4eb9-bf3c-b623cde687e3 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.251310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 656/1302] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:12:18.256885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.257366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.257483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.273830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:18.274998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:18.339151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 647/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:12:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 09:12:18.361991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa22c-8323-4eb9-bf3c-b623cde687e3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:18.455152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:18.461610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa22c-8323-4eb9-bf3c-b623cde687e3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:18.495409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:18.496198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.497034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.497034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.497411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:18.498771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 09:12:18.510795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 09:12:18.511896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application with Lock( Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self._enter() Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application return self.creator() Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application created_value = creator( Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: ERROR keystone.server.flask.application Jun 07 09:12:18.512880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 657/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Jun 7 09:12:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 09:12:18.518211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:18.592789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:18.615735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce7087-cff4-48b9-a46d-ea40a803b434 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:18.648893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:18.649571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.649854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.650127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] PATH_INFO: `/v3/services` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.650828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:18.651134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:18.653054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429549b3-af15-44d8-8ccb-d024d4012e72 None admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:18.663516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 648/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Jun 7 09:12:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:18.664041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:18.760907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb5d7e-004b-43bf-9e91-e6eaeca4ed4b None admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:18.793963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:18.794648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.794933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.795209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.795942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:18.797338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f80a3-e9d8-4a33-a85a-390b10fc86b4 None admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:18.881706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 658/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Jun 7 09:12:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:28.655152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=30 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:28.876961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:40.420317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6601d9-6f77-40fb-a1af-d4cad9eb5725 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:40.420863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6601d9-6f77-40fb-a1af-d4cad9eb5725 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:40.420863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6601d9-6f77-40fb-a1af-d4cad9eb5725 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:40.427543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65666-05a2-40b0-89aa-896912f3dcd9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:40.428025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65666-05a2-40b0-89aa-896912f3dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:40.428126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65666-05a2-40b0-89aa-896912f3dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:40.431198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 649/1307] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:40.513447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bbe65666-05a2-40b0-89aa-896912f3dcd9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:40.514606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bbe65666-05a2-40b0-89aa-896912f3dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:40.661769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 659/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:19:40.689136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:40.689498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:40.689891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:40.691861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 650/1309] 10.222.0.57 () {54 vars in 964 bytes} [Tue Jun 7 09:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:40.707210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6142ad-ca48-4520-b6b4-0388409469ca None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:40.707501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6142ad-ca48-4520-b6b4-0388409469ca None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:40.707776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6142ad-ca48-4520-b6b4-0388409469ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:40.802409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-fc6142ad-ca48-4520-b6b4-0388409469ca None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:40.803726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fc6142ad-ca48-4520-b6b4-0388409469ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:40.827597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:40.911465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 660/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Jun 7 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:40.920879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:40.922657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:41.057809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8672b9-4272-4cb6-92e7-0c358b231002 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:41.135759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:41.137089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:41.137886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:41.138415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:41.140070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:41.198809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:41.255375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:41.256822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a34921-6d3e-4f0f-a670-d1218b34ef96 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:41.276325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 651/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:19:43.789800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:43.790329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:43.790329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:43.791959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 661/1312] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:43.798855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49881-0550-44fa-8240-0b5b4c5a367a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:43.799339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49881-0550-44fa-8240-0b5b4c5a367a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:43.799433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49881-0550-44fa-8240-0b5b4c5a367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:43.916265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-75f49881-0550-44fa-8240-0b5b4c5a367a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:43.917595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-75f49881-0550-44fa-8240-0b5b4c5a367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:44.025344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 652/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:44.063029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:44.064621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:44.169365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98402-bdbc-4dcc-b2b4-308ad6466e8c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:44.239242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:44.241869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:44.242518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:44.242801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:44.243077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:44.244766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:44.357901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:44.359470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddcf2c-e1ef-4bf0-abc2-9073f44f2a69 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:44.379915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 662/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:19:47.148459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:47.149063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:47.149063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:47.150472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 653/1315] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:47.156213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7be2-339b-4177-8a9c-f29d6e0eb22f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:47.156727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7be2-339b-4177-8a9c-f29d6e0eb22f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:47.156830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7be2-339b-4177-8a9c-f29d6e0eb22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:47.220806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-fe1a7be2-339b-4177-8a9c-f29d6e0eb22f None None] MFA Rules not processed for user `9d949c21d8c14c66b498639dd44ff225`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:47.222003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fe1a7be2-339b-4177-8a9c-f29d6e0eb22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:47.272506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:47.315756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 663/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:47.355095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:47.356687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:47.366287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:47.465597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35256dd-fdeb-46bc-a108-35f0729b0995 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:47.537406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:47.537983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:47.538268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:47.538542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:47.540192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:47.602009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:47.649108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:47.650522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3547a-e6d1-4f82-a74f-f7bf0849101e service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:47.669293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 654/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:19:50.744782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17dc7-630d-4242-bc20-01b2329cfaf3 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:50.745292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17dc7-630d-4242-bc20-01b2329cfaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:50.745292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17dc7-630d-4242-bc20-01b2329cfaf3 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:50.747498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 664/1318] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:50.752877 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f0d7a-5c34-4bcf-ba79-99a63ec88347 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:50.753301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f0d7a-5c34-4bcf-ba79-99a63ec88347 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:50.753410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f0d7a-5c34-4bcf-ba79-99a63ec88347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:50.816453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-870f0d7a-5c34-4bcf-ba79-99a63ec88347 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:50.817655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-870f0d7a-5c34-4bcf-ba79-99a63ec88347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:50.911741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 655/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:50.966355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:50.966682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:50.966966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:50.968353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 665/1320] 10.222.0.57 () {54 vars in 964 bytes} [Tue Jun 7 09:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:50.974130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031d3b-bf95-48f6-a35c-f1ab5c291ee0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:50.974418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031d3b-bf95-48f6-a35c-f1ab5c291ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:50.974693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031d3b-bf95-48f6-a35c-f1ab5c291ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:50.994888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:51.049252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1f031d3b-bf95-48f6-a35c-f1ab5c291ee0 None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:51.050420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1f031d3b-bf95-48f6-a35c-f1ab5c291ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:51.124008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 656/1321] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Jun 7 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:51.133038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:51.134575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:51.237098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ae00b-202c-40f9-a425-f8e2cda47e97 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:51.310275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:51.310843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:51.311126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:51.311403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:51.313079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:51.319834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:51.420758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:51.422208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38847-fc4d-4e7b-b98d-d01fea3c9b5d service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:51.439396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 666/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:19:55.057464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:55.058009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:55.058009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:55.059970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 657/1323] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:55.065157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f735e-7607-4c65-98d2-0f01bb6e2642 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:55.065560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f735e-7607-4c65-98d2-0f01bb6e2642 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:55.065713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f735e-7607-4c65-98d2-0f01bb6e2642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:55.130066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-208f735e-7607-4c65-98d2-0f01bb6e2642 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:55.131272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-208f735e-7607-4c65-98d2-0f01bb6e2642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:55.187714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:55.226486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 667/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:55.263279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:55.264814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:55.366799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9678a0-c4b7-4f61-87ad-6bf2629f4555 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:55.379989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:55.442163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:55.442806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:55.443087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:55.443365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:55.445035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:55.553652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:55.555087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade75fe-df2a-45ee-b4ba-e8664d858580 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:55.573784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 658/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:19:57.902050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:57.903204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:57.903320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:57.904668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 668/1326] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:19:57.910190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdd2e-f0aa-4117-9db4-d2eb9d3dd6bd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:57.910616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdd2e-f0aa-4117-9db4-d2eb9d3dd6bd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:57.910747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdd2e-f0aa-4117-9db4-d2eb9d3dd6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:57.958936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:57.977320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c28cdd2e-f0aa-4117-9db4-d2eb9d3dd6bd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:57.978537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c28cdd2e-f0aa-4117-9db4-d2eb9d3dd6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:58.078154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 659/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:58.114302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:19:58.115950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:58.225307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d461a-dad8-445b-9e79-b90a58136042 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:19:58.300760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:19:58.301416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:58.301699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:58.301970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:58.303602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:58.320799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:58.548600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:19:58.550112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304057f7-e076-453e-8eae-6d5eecae517c service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:19:58.569307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 669/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:00.848991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:00.849563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:00.849563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:00.850932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 660/1329] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:00.856147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ab98-d225-4e54-824e-d8c007bca3e5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:00.856683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ab98-d225-4e54-824e-d8c007bca3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:00.856874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ab98-d225-4e54-824e-d8c007bca3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:00.920363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a084ab98-d225-4e54-824e-d8c007bca3e5 None None] MFA Rules not processed for user `9d949c21d8c14c66b498639dd44ff225`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:00.921607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a084ab98-d225-4e54-824e-d8c007bca3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:00.985783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:01.019679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 670/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:01.059223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:01.061209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:01.170531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e2cd-254f-4e8e-bcbf-70adc1705de2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:01.190752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:01.243917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:01.244554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:01.244861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:01.245129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:01.247348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:01.355185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:01.357290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2ae2d-58f8-4b58-a9e6-0967e6aa2090 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:01.375976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 661/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:04.073081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:04.073624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:04.073624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:04.075590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 671/1332] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:04.081121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6ed6-9b17-4ae9-84d5-6e249a5f5b16 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:04.081566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6ed6-9b17-4ae9-84d5-6e249a5f5b16 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:04.081679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6ed6-9b17-4ae9-84d5-6e249a5f5b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:04.145948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:04.148700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a02c6ed6-9b17-4ae9-84d5-6e249a5f5b16 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:04.149982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a02c6ed6-9b17-4ae9-84d5-6e249a5f5b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:04.247654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 662/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:04.282222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:04.283742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:04.387568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f1379-c882-45fd-bd82-0e6056d48e2c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:04.464610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:04.465222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:04.465500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:04.465771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:04.467404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:04.495677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:04.594567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:04.596119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ae449-3847-4f38-9213-c01311c944bd service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:04.617202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 672/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:06.926378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:06.926903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:06.926903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:06.928336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 663/1335] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:06.934109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059284-7cc8-4ed7-903a-9eb5e96888f5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:06.934512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059284-7cc8-4ed7-903a-9eb5e96888f5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:06.934657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059284-7cc8-4ed7-903a-9eb5e96888f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:06.998013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-62059284-7cc8-4ed7-903a-9eb5e96888f5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:06.999213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-62059284-7cc8-4ed7-903a-9eb5e96888f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:07.072981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:07.094649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 673/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:07.134847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:07.136373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:07.237358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc55e1-0e8c-4e15-aa59-69ae687095d2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:07.266143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:07.310319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:07.310954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:07.311236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:07.311509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:07.313156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:07.421028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:07.422457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d7e8a-3003-4e71-9150-c6397923c5aa service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:07.440474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 664/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:10.268132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:10.268688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:10.268688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:10.270001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 674/1338] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:10.275084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527ace2-67f3-4d36-a439-060e953d1f10 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:10.275464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527ace2-67f3-4d36-a439-060e953d1f10 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:10.275648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527ace2-67f3-4d36-a439-060e953d1f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:10.340090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7527ace2-67f3-4d36-a439-060e953d1f10 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:10.341318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7527ace2-67f3-4d36-a439-060e953d1f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:10.349506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:10.437310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 665/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:10.473067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:10.476021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:10.582168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfde8d-3f21-4b0f-8abb-066ee31b81b6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:10.594094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:10.594727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:10.595006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:10.595276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:10.596876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:10.695157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:10.697500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:10.699513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952a0ba-5787-4cf2-821e-2f5fb348d1d3 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:10.719064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 675/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:15.247918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:15.248471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:15.248471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:15.249801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 666/1341] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:15.255176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d7e9-2177-4c23-9097-32258f98e82f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:15.255637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d7e9-2177-4c23-9097-32258f98e82f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:15.255895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d7e9-2177-4c23-9097-32258f98e82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:15.307379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5f70d7e9-2177-4c23-9097-32258f98e82f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:15.309297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5f70d7e9-2177-4c23-9097-32258f98e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:15.387653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 676/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:15.426665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:15.428800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:15.452200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22d18e-7cb6-413e-9a78-b84bdd424066 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:15.455712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:15.456342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:15.456646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:15.456920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:15.465806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:15.467255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da09617-9879-416b-b452-251f47ad9c6c service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:15.476866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 667/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:18.649844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ca0f-eaaa-4e52-a8d8-5fbff190f3d4 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:18.650383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ca0f-eaaa-4e52-a8d8-5fbff190f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:18.650383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ca0f-eaaa-4e52-a8d8-5fbff190f3d4 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:18.652236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 677/1344] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:18.657648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349fbc-4b19-4c4b-8f2a-92feab183d14 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:18.658080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349fbc-4b19-4c4b-8f2a-92feab183d14 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:18.658312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349fbc-4b19-4c4b-8f2a-92feab183d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:18.685638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5c349fbc-4b19-4c4b-8f2a-92feab183d14 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:18.686803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5c349fbc-4b19-4c4b-8f2a-92feab183d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:18.694417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 668/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:18.732019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650ca0f-eaaa-4e52-a8d8-5fbff190f3d4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:18.742547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650ca0f-eaaa-4e52-a8d8-5fbff190f3d4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:18.746222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:18.746855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:18.747147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:18.747400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:18.756269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:18.757730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ee9f3-c205-4d8c-b1d0-2015ecf83387 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:18.767177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 678/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:23.378297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b80fd-3274-4632-9050-90436dd7b056 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:23.378832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b80fd-3274-4632-9050-90436dd7b056 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:23.378832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b80fd-3274-4632-9050-90436dd7b056 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:23.380177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 669/1347] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:23.385475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356a7b0-5073-49bf-ac0c-cb6f4f5627f2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:23.385953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356a7b0-5073-49bf-ac0c-cb6f4f5627f2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:23.386061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356a7b0-5073-49bf-ac0c-cb6f4f5627f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:23.405415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6356a7b0-5073-49bf-ac0c-cb6f4f5627f2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:23.406594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6356a7b0-5073-49bf-ac0c-cb6f4f5627f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:23.414310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 679/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:23.451885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b80fd-3274-4632-9050-90436dd7b056 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:23.462747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b80fd-3274-4632-9050-90436dd7b056 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:23.466778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:23.467405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:23.467685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:23.467958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:23.477096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:23.478508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d861f-6569-4fce-b583-2e3455ebb12b service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:23.489005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 670/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:28.859064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d9ac-53ca-4ee5-a5e0-2a69c444d699 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:28.859967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d9ac-53ca-4ee5-a5e0-2a69c444d699 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:28.860542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d9ac-53ca-4ee5-a5e0-2a69c444d699 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:28.862115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 680/1350] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:28.868490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011eafa-e1a6-4789-8449-0152e146f964 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:28.875746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011eafa-e1a6-4789-8449-0152e146f964 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:28.875746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011eafa-e1a6-4789-8449-0152e146f964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:28.986533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d011eafa-e1a6-4789-8449-0152e146f964 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:28.988216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d011eafa-e1a6-4789-8449-0152e146f964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:29.051685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:29.122610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 671/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:33.354651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0cb5e-bde8-4a8b-a368-8396a93b985d None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:33.355715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0cb5e-bde8-4a8b-a368-8396a93b985d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:33.356319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0cb5e-bde8-4a8b-a368-8396a93b985d None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:33.358365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 681/1352] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:20:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:33.364051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6ccd-6eb1-440a-ae68-c5839055d200 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:33.364957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6ccd-6eb1-440a-ae68-c5839055d200 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:33.365427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6ccd-6eb1-440a-ae68-c5839055d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:33.389851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-af6f6ccd-6eb1-440a-ae68-c5839055d200 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:33.391366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-af6f6ccd-6eb1-440a-ae68-c5839055d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:33.398512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=60 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:33.405886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 672/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:20:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:34.974609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1b9a-11cc-4a1e-941d-b1e549d180b0 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:34.975147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1b9a-11cc-4a1e-941d-b1e549d180b0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:34.975326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1b9a-11cc-4a1e-941d-b1e549d180b0 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:34.976926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 682/1354] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:34.982676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5421b9f-f162-4412-8925-f8e81fbe3318 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:34.983108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5421b9f-f162-4412-8925-f8e81fbe3318 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:34.983297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5421b9f-f162-4412-8925-f8e81fbe3318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:35.003053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f5421b9f-f162-4412-8925-f8e81fbe3318 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:35.004258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f5421b9f-f162-4412-8925-f8e81fbe3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:35.013872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 673/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:20:38.840948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e30b6-f231-4ed3-b825-4b7a1b00d52d None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:38.841470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e30b6-f231-4ed3-b825-4b7a1b00d52d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:38.841470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e30b6-f231-4ed3-b825-4b7a1b00d52d None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:38.842871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 683/1356] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:38.848162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad15f-d7c6-4efb-9c03-23c691cf6306 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:38.848639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad15f-d7c6-4efb-9c03-23c691cf6306 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:38.848772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbad15f-d7c6-4efb-9c03-23c691cf6306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:38.867738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5bbad15f-d7c6-4efb-9c03-23c691cf6306 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:38.868998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5bbad15f-d7c6-4efb-9c03-23c691cf6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:38.878118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 674/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:38.932906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef0cf-1864-4947-a334-365cb5df3fab None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:38.933496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef0cf-1864-4947-a334-365cb5df3fab None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:38.934304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef0cf-1864-4947-a334-365cb5df3fab None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:38.935904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 684/1358] 10.222.0.57 () {54 vars in 961 bytes} [Tue Jun 7 09:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:38.946219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1f43-4d8d-4b04-9f65-ef2e5b1d481e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:38.947052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1f43-4d8d-4b04-9f65-ef2e5b1d481e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:38.947875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1f43-4d8d-4b04-9f65-ef2e5b1d481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:39.009520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ff2e1f43-4d8d-4b04-9f65-ef2e5b1d481e None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:39.010826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ff2e1f43-4d8d-4b04-9f65-ef2e5b1d481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:39.102953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 675/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Jun 7 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:39.110367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eef0cf-1864-4947-a334-365cb5df3fab None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:39.127661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eef0cf-1864-4947-a334-365cb5df3fab None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:39.134085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:39.135052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:39.135555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:39.135981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:39.146010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:39.148125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243800-7534-48c0-992b-e00874f35db3 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:39.161543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 685/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:41.611489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f3b0-4c1a-4827-a918-92fbb913340b None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:41.612028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f3b0-4c1a-4827-a918-92fbb913340b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:41.612028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f3b0-4c1a-4827-a918-92fbb913340b None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:41.613465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 676/1361] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:41.618704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004398-94f5-4214-a5b1-08ea40efe6f9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:41.619150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004398-94f5-4214-a5b1-08ea40efe6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:41.619260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004398-94f5-4214-a5b1-08ea40efe6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:41.637974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-56004398-94f5-4214-a5b1-08ea40efe6f9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:41.639156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-56004398-94f5-4214-a5b1-08ea40efe6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:41.647928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 686/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:41.691804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091f3b0-4c1a-4827-a918-92fbb913340b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:41.706464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091f3b0-4c1a-4827-a918-92fbb913340b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:41.711951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:41.712603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:41.713108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:41.713388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:41.721752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:41.723326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee942-53ca-46be-a35c-52338ff27baa service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:41.733474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 677/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:44.582966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e604-17fe-4c9a-9280-359fcd40fe36 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:44.583517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e604-17fe-4c9a-9280-359fcd40fe36 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:44.583517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e604-17fe-4c9a-9280-359fcd40fe36 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:44.584912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 687/1364] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:44.590089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd49f2-4300-4ab1-82fb-1e41cd0ddd55 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:44.590513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd49f2-4300-4ab1-82fb-1e41cd0ddd55 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:44.590710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd49f2-4300-4ab1-82fb-1e41cd0ddd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:44.608734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-67fd49f2-4300-4ab1-82fb-1e41cd0ddd55 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:44.609905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-67fd49f2-4300-4ab1-82fb-1e41cd0ddd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:44.617801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 678/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:20:44.663623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba363-48cb-4e48-80dd-ca47c3744b75 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:44.663914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba363-48cb-4e48-80dd-ca47c3744b75 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:44.664194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba363-48cb-4e48-80dd-ca47c3744b75 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:44.665641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 688/1366] 10.222.0.57 () {54 vars in 961 bytes} [Tue Jun 7 09:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:44.671392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5120-ef0c-4eea-a506-39540182e9e5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:44.671677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5120-ef0c-4eea-a506-39540182e9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:44.671955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5120-ef0c-4eea-a506-39540182e9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:44.691029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8a1a5120-ef0c-4eea-a506-39540182e9e5 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:44.692335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8a1a5120-ef0c-4eea-a506-39540182e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:44.701544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 679/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Jun 7 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:20:44.709318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba363-48cb-4e48-80dd-ca47c3744b75 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:44.723148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba363-48cb-4e48-80dd-ca47c3744b75 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:44.728487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:44.729618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:44.729895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:44.730166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:44.738094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:44.739667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e9d49-f15b-4b1c-8592-44802cbf11ec service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:44.750117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 689/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:47.156879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4f21f-0e93-4f79-9d9d-edd796a5263b None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.157745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4f21f-0e93-4f79-9d9d-edd796a5263b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.158027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4f21f-0e93-4f79-9d9d-edd796a5263b None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.159661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 680/1369] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:47.164775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4cb68-ef06-465c-abdc-f788e604e521 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.165211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4cb68-ef06-465c-abdc-f788e604e521 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.165326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4cb68-ef06-465c-abdc-f788e604e521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.183784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2cc4cb68-ef06-465c-abdc-f788e604e521 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:47.185308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2cc4cb68-ef06-465c-abdc-f788e604e521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:47.196938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 690/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:47.340385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e99f-3ed6-4eb6-8ee8-649ecf7a1827 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.340570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e99f-3ed6-4eb6-8ee8-649ecf7a1827 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.340950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e99f-3ed6-4eb6-8ee8-649ecf7a1827 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.342318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 681/1371] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:47.347659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968f47-c45c-4317-b438-9d0214e33756 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.347947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968f47-c45c-4317-b438-9d0214e33756 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.348223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968f47-c45c-4317-b438-9d0214e33756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.366441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5d968f47-c45c-4317-b438-9d0214e33756 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:47.367616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5d968f47-c45c-4317-b438-9d0214e33756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:47.375831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 691/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:47.420225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31e99f-3ed6-4eb6-8ee8-649ecf7a1827 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:47.435279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31e99f-3ed6-4eb6-8ee8-649ecf7a1827 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:47.440801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:47.441434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.441741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.442252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.450847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:47.452420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e53d75-0990-46d4-b55e-2e396593e836 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:47.462644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 682/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:47.497256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145dfb-88e0-4680-9bd1-0545bf81cd5f None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.497540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145dfb-88e0-4680-9bd1-0545bf81cd5f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.497810 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145dfb-88e0-4680-9bd1-0545bf81cd5f None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.499178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 692/1374] 10.222.0.57 () {54 vars in 963 bytes} [Tue Jun 7 09:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:47.504689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5429d6-fc9b-4edd-8ef5-9385848081d5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.504984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5429d6-fc9b-4edd-8ef5-9385848081d5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.505262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5429d6-fc9b-4edd-8ef5-9385848081d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.535058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:47.554005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fc5429d6-fc9b-4edd-8ef5-9385848081d5 None None] MFA Rules not processed for user `c7d851c1cb8b495a87692b8e8970f851`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:47.555216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fc5429d6-fc9b-4edd-8ef5-9385848081d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:47.622747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 683/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Tue Jun 7 09:20:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:47.630365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2145dfb-88e0-4680-9bd1-0545bf81cd5f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:47.640643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2145dfb-88e0-4680-9bd1-0545bf81cd5f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:47.644067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:47.644713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:47.644995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:47.645263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:47.654078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:47.655488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e8d3c-7a3f-4289-87c4-6a5f6f4a06b4 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:47.665252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 693/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:50.010975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ced4-b7b8-4908-9a33-1e8fc8e5904c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:50.011420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ced4-b7b8-4908-9a33-1e8fc8e5904c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:50.011551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ced4-b7b8-4908-9a33-1e8fc8e5904c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:50.013135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 684/1377] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:50.019313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bac89-c057-499f-bafb-602f8eb1fe22 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:50.019806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bac89-c057-499f-bafb-602f8eb1fe22 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:50.019917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bac89-c057-499f-bafb-602f8eb1fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:50.038052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-117bac89-c057-499f-bafb-602f8eb1fe22 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:50.038985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-117bac89-c057-499f-bafb-602f8eb1fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:50.047651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 694/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:50.092665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11ced4-b7b8-4908-9a33-1e8fc8e5904c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:50.106956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11ced4-b7b8-4908-9a33-1e8fc8e5904c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:50.112726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:50.113365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:50.113652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:50.113929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:50.123104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:50.124730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774c9f-978d-4b93-a0f2-2f2d0186ebdf service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:50.135467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 685/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:52.543355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd556d3-aa61-4b0c-a7bb-7f7da050d6cd None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:52.544274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd556d3-aa61-4b0c-a7bb-7f7da050d6cd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:52.544686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd556d3-aa61-4b0c-a7bb-7f7da050d6cd None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:52.546295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 695/1380] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:20:52.551749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4da63-6b3b-4582-848e-27a9a114d62f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:52.552217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4da63-6b3b-4582-848e-27a9a114d62f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:52.552318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4da63-6b3b-4582-848e-27a9a114d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:52.571565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-eee4da63-6b3b-4582-848e-27a9a114d62f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:52.573102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eee4da63-6b3b-4582-848e-27a9a114d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:52.581805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 686/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:20:52.626642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd556d3-aa61-4b0c-a7bb-7f7da050d6cd None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:52.641734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd556d3-aa61-4b0c-a7bb-7f7da050d6cd None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:52.647296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:52.648040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:52.648426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:52.648827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:52.658246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:52.659960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0254aa-ca3a-4e0a-aea9-c373f31dd1d8 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:52.670376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 696/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:55.065606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421b355-6bda-4d95-98e4-2c357dad422e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:55.066168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421b355-6bda-4d95-98e4-2c357dad422e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:55.066168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421b355-6bda-4d95-98e4-2c357dad422e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:55.067520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 687/1383] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:55.073683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94230-a50e-4bda-bba6-e62f8363da6d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:55.074522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94230-a50e-4bda-bba6-e62f8363da6d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:55.074880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94230-a50e-4bda-bba6-e62f8363da6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:55.093811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-39e94230-a50e-4bda-bba6-e62f8363da6d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:55.095095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-39e94230-a50e-4bda-bba6-e62f8363da6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:55.103634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 697/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:20:55.145872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421b355-6bda-4d95-98e4-2c357dad422e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:55.161554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421b355-6bda-4d95-98e4-2c357dad422e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:55.167146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:55.167865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:55.168339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:55.168716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:55.177813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:55.179525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f248b8-2450-4146-9051-2f5717331f36 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:55.190001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 688/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:57.953121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbb9c-f63c-4404-bed2-ededaf3d7a8f None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:57.953687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbb9c-f63c-4404-bed2-ededaf3d7a8f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:57.953889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbb9c-f63c-4404-bed2-ededaf3d7a8f None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:57.955935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 698/1386] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:20:57.963203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3075dca-8ad1-4141-bdaa-b45008e97991 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:57.963714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3075dca-8ad1-4141-bdaa-b45008e97991 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:57.963714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3075dca-8ad1-4141-bdaa-b45008e97991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:57.982376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a3075dca-8ad1-4141-bdaa-b45008e97991 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:20:57.983557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a3075dca-8ad1-4141-bdaa-b45008e97991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:57.991384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 689/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:20:58.047830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bbb9c-f63c-4404-bed2-ededaf3d7a8f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:58.066086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bbb9c-f63c-4404-bed2-ededaf3d7a8f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:58.075944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:58.076759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:58.077223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:58.077586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:58.089432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:58.091702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272427f-e0bc-4e7e-9c99-dfc1692d6d7f service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:58.104277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 699/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:00.584979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875e306-a742-46ef-8c86-ba980621477a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:00.585594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875e306-a742-46ef-8c86-ba980621477a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:00.585594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875e306-a742-46ef-8c86-ba980621477a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:00.586880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 690/1389] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:00.592674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956066ae-d468-472d-af1e-22ded7c448d9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:00.593088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956066ae-d468-472d-af1e-22ded7c448d9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:00.593209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956066ae-d468-472d-af1e-22ded7c448d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:00.611648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-956066ae-d468-472d-af1e-22ded7c448d9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:00.613161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-956066ae-d468-472d-af1e-22ded7c448d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:00.621593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 700/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:00.666563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875e306-a742-46ef-8c86-ba980621477a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:00.687118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875e306-a742-46ef-8c86-ba980621477a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:00.694133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:00.694806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:00.695088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:00.695362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:00.705333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:00.706984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6f261-0095-46ed-b386-c7761d02908f service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:00.719453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 691/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:03.259650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599610-ee57-43e6-8110-c7402c07c4ee None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.260222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599610-ee57-43e6-8110-c7402c07c4ee None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.260222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599610-ee57-43e6-8110-c7402c07c4ee None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.261710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 701/1392] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:03.266827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc641-ca86-472d-a3a2-c8d2493660c7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.267263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc641-ca86-472d-a3a2-c8d2493660c7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.267411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc641-ca86-472d-a3a2-c8d2493660c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.285185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e74fc641-ca86-472d-a3a2-c8d2493660c7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:03.286366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e74fc641-ca86-472d-a3a2-c8d2493660c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:03.294871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 692/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:03.338840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599610-ee57-43e6-8110-c7402c07c4ee None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:03.353751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599610-ee57-43e6-8110-c7402c07c4ee None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:03.359173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:03.359806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.360094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.360371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.369465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:03.371342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df19ea-5010-49ac-ab10-bfaabcebf2a0 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:03.380973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 702/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:03.842932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afcd8-7c96-4cf1-8a57-da2fd2765dec None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.843166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afcd8-7c96-4cf1-8a57-da2fd2765dec None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.843419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afcd8-7c96-4cf1-8a57-da2fd2765dec None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.844877 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 693/1395] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:03.850278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd013-5faa-469e-b25b-710420731d7e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.850570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd013-5faa-469e-b25b-710420731d7e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.850849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd013-5faa-469e-b25b-710420731d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.857532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=55 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:03.869341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bdbcd013-5faa-469e-b25b-710420731d7e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:03.870793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bdbcd013-5faa-469e-b25b-710420731d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:03.878641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 703/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:03.934294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd4b-743b-4510-b1c5-a2ad6c711037 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.934588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd4b-743b-4510-b1c5-a2ad6c711037 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.934963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd4b-743b-4510-b1c5-a2ad6c711037 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.936385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 694/1397] 10.222.0.57 () {54 vars in 962 bytes} [Tue Jun 7 09:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:03.941687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff5e5f-917b-4ef7-9fd8-479f052ea869 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:03.942165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff5e5f-917b-4ef7-9fd8-479f052ea869 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:03.942442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff5e5f-917b-4ef7-9fd8-479f052ea869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:03.990221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-88ff5e5f-917b-4ef7-9fd8-479f052ea869 None None] MFA Rules not processed for user `7025724d5c31460db223411aab77003a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:03.991430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-88ff5e5f-917b-4ef7-9fd8-479f052ea869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:04.051536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 704/1398] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Jun 7 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:04.059050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fd4b-743b-4510-b1c5-a2ad6c711037 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:04.069790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fd4b-743b-4510-b1c5-a2ad6c711037 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:04.073802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:04.074433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:04.074706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:04.074976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:04.083800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:04.087853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c528bd-6809-4801-845e-ebaa83eb3752 service glance] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:04.095224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 695/1399] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:04.764165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb6ae-69a0-41aa-aeb6-44d709c12ee4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:04.764696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb6ae-69a0-41aa-aeb6-44d709c12ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:04.764887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb6ae-69a0-41aa-aeb6-44d709c12ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:04.816268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a08eb6ae-69a0-41aa-aeb6-44d709c12ee4 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:04.817514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a08eb6ae-69a0-41aa-aeb6-44d709c12ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:04.884979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 705/1400] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:21:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:04.926918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d0f-f0e1-4109-bb6d-5475c352536d None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:04.927415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d0f-f0e1-4109-bb6d-5475c352536d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:04.927717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d0f-f0e1-4109-bb6d-5475c352536d None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:04.929253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 696/1401] 10.222.0.57 () {54 vars in 942 bytes} [Tue Jun 7 09:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:04.935674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f903-d969-4535-8d8b-06cb2e4de75d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:04.936164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f903-d969-4535-8d8b-06cb2e4de75d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:04.936354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f903-d969-4535-8d8b-06cb2e4de75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:04.985848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-c0b1f903-d969-4535-8d8b-06cb2e4de75d None None] MFA Rules not processed for user `327a3409391e4d558aeab965401f48e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:04.987180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c0b1f903-d969-4535-8d8b-06cb2e4de75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:05.000393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:05.059940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 706/1402] 10.222.0.57 () {58 vars in 1039 bytes} [Tue Jun 7 09:21:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:05.069344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071d0f-f0e1-4109-bb6d-5475c352536d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:05.083581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071d0f-f0e1-4109-bb6d-5475c352536d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:05.089170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:05.089802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:05.090082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:05.090354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:05.098471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:05.100030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856e59-0626-4147-b013-16d0bb076c6d service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:05.108081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 697/1403] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:21:05.791868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff407-0aac-41bc-a6ef-1f2ede5cfd67 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:05.792441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff407-0aac-41bc-a6ef-1f2ede5cfd67 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:05.792441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff407-0aac-41bc-a6ef-1f2ede5cfd67 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:05.794300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 707/1404] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:05.799269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee73a1-8c15-4bd5-8ac9-19162e2a649e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:05.799554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee73a1-8c15-4bd5-8ac9-19162e2a649e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:05.799827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee73a1-8c15-4bd5-8ac9-19162e2a649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:05.819482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-85ee73a1-8c15-4bd5-8ac9-19162e2a649e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:05.823406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-85ee73a1-8c15-4bd5-8ac9-19162e2a649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:05.832567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 698/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:05.878011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff407-0aac-41bc-a6ef-1f2ede5cfd67 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:05.896567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff407-0aac-41bc-a6ef-1f2ede5cfd67 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:05.902727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:05.903367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:05.903657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:05.903930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:05.913417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:05.915011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5180-50ea-46d5-822d-b94fdc1c0bb2 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:05.930274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 708/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:08.512966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7ed4-0c11-4863-8594-48d5b37cd55b None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:08.513503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7ed4-0c11-4863-8594-48d5b37cd55b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:08.513503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7ed4-0c11-4863-8594-48d5b37cd55b None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:08.515025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 699/1407] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:08.520268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f3a2b-941a-4f7c-9896-dd0d96fdd80c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:08.520723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f3a2b-941a-4f7c-9896-dd0d96fdd80c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:08.520896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f3a2b-941a-4f7c-9896-dd0d96fdd80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:08.540006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-694f3a2b-941a-4f7c-9896-dd0d96fdd80c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:08.543611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-694f3a2b-941a-4f7c-9896-dd0d96fdd80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:08.552501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 709/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:08.596903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7ed4-0c11-4863-8594-48d5b37cd55b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:08.614476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7ed4-0c11-4863-8594-48d5b37cd55b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:08.621435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:08.622150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:08.622533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:08.622889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:08.631978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:08.635776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c80a7-d200-4d8a-b4fb-9bf25dbe0139 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:08.646529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 700/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:18.535841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=33 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:18.643531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=76 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:29.762902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d7c1-2bca-4c93-89c2-443a1aec8d96 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:29.763544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d7c1-2bca-4c93-89c2-443a1aec8d96 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:29.763544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d7c1-2bca-4c93-89c2-443a1aec8d96 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:29.764890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 710/1410] 10.222.0.57 () {54 vars in 918 bytes} [Tue Jun 7 09:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:29.769943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcc2c-dd70-4e98-9c5a-7dc1e921f244 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:29.770405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcc2c-dd70-4e98-9c5a-7dc1e921f244 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:29.770514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dcc2c-dd70-4e98-9c5a-7dc1e921f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:29.788826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-249dcc2c-dd70-4e98-9c5a-7dc1e921f244 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:29.790012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-249dcc2c-dd70-4e98-9c5a-7dc1e921f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:29.799125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 701/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Jun 7 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:29.813030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ec3a5-f40e-46ec-befa-347d61bb36c2 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:29.813317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ec3a5-f40e-46ec-befa-347d61bb36c2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:29.813592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ec3a5-f40e-46ec-befa-347d61bb36c2 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:29.814967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 711/1412] 10.222.0.57 () {54 vars in 918 bytes} [Tue Jun 7 09:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:29.819506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656044-b146-444a-8e56-801af69df152 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:29.819800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656044-b146-444a-8e56-801af69df152 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:29.820359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656044-b146-444a-8e56-801af69df152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:29.839306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-05656044-b146-444a-8e56-801af69df152 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:29.840661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-05656044-b146-444a-8e56-801af69df152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:29.849579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 702/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Jun 7 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:21:32.546249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675de35-ea6f-457d-a1c0-4977d7b2246d None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:32.546678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675de35-ea6f-457d-a1c0-4977d7b2246d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:32.546781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675de35-ea6f-457d-a1c0-4977d7b2246d None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:32.548167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:32.553528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f052-b90f-4b79-b872-de2344cd14f7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:32.553942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f052-b90f-4b79-b872-de2344cd14f7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:32.554081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f052-b90f-4b79-b872-de2344cd14f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:32.572227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c245f052-b90f-4b79-b872-de2344cd14f7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:32.573540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c245f052-b90f-4b79-b872-de2344cd14f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:32.581236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:21:32.625209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675de35-ea6f-457d-a1c0-4977d7b2246d None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:32.639792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675de35-ea6f-457d-a1c0-4977d7b2246d None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:32.645273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:32.645898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:32.646180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:32.646452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:32.655187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:32.656795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9166a-1ec2-40fc-8712-22a998011119 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:32.666229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 713/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:33.632601 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1089a0e-4522-466d-857a-071f9c53ea7a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:33.633661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1089a0e-4522-466d-857a-071f9c53ea7a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:33.633661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1089a0e-4522-466d-857a-071f9c53ea7a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:33.635036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:33.639831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8cb64-ccdf-40f5-a584-4ab1756c24a7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:33.640249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8cb64-ccdf-40f5-a584-4ab1756c24a7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:33.640379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8cb64-ccdf-40f5-a584-4ab1756c24a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:33.658416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8cb8cb64-ccdf-40f5-a584-4ab1756c24a7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:33.659614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8cb8cb64-ccdf-40f5-a584-4ab1756c24a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:33.667796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:33.842641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1089a0e-4522-466d-857a-071f9c53ea7a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:33.857769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1089a0e-4522-466d-857a-071f9c53ea7a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:33.863434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:33.864101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:33.864362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:33.864664 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:33.874343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:33.875910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333614-06c3-43aa-968f-4c02ecd909f3 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:33.885824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 705/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:34.815013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193870-8192-4daf-b47f-aad7474e8e73 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:34.815503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193870-8192-4daf-b47f-aad7474e8e73 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:34.815503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193870-8192-4daf-b47f-aad7474e8e73 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:34.816943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:34.821807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9cf35-8f9b-4b70-a830-43fed169bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:34.827285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9cf35-8f9b-4b70-a830-43fed169bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:34.827285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9cf35-8f9b-4b70-a830-43fed169bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:34.840000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2ad9cf35-8f9b-4b70-a830-43fed169bcb6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:34.841228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2ad9cf35-8f9b-4b70-a830-43fed169bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:34.849000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:35.000225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193870-8192-4daf-b47f-aad7474e8e73 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:35.015433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193870-8192-4daf-b47f-aad7474e8e73 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:35.020832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:35.021488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:35.021771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:35.022038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:35.030349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:35.031911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a476144-7e4f-4f87-998f-c73729a7e0e1 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:35.041870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 716/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:35.933801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86778-8b58-4e8a-879a-2d8c29262bea None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:35.934359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86778-8b58-4e8a-879a-2d8c29262bea None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:35.934359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86778-8b58-4e8a-879a-2d8c29262bea None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:35.935763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:35.940791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25f29-f768-48fb-b451-2ceef2c93b3f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:35.941252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25f29-f768-48fb-b451-2ceef2c93b3f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:35.941363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25f29-f768-48fb-b451-2ceef2c93b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:35.959733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-93c25f29-f768-48fb-b451-2ceef2c93b3f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:35.960990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-93c25f29-f768-48fb-b451-2ceef2c93b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:35.968903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:36.118137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86778-8b58-4e8a-879a-2d8c29262bea None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:36.132423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86778-8b58-4e8a-879a-2d8c29262bea None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:36.137667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:36.138286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:36.138565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:36.138835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:36.147038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:36.149161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01c395-092f-4268-b6cc-ee4fc476fa62 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:36.158460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 708/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:21:37.061818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8dd0-0d68-406f-aee6-6a4f0eb40703 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:37.062344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8dd0-0d68-406f-aee6-6a4f0eb40703 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:37.062344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8dd0-0d68-406f-aee6-6a4f0eb40703 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:37.063748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:37.068615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57436-af9e-438f-96e9-5752c88ba4ab None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:37.069063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57436-af9e-438f-96e9-5752c88ba4ab None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:37.069170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57436-af9e-438f-96e9-5752c88ba4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:37.088217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-08f57436-af9e-438f-96e9-5752c88ba4ab None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:37.089510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-08f57436-af9e-438f-96e9-5752c88ba4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:37.097277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:37.244726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e8dd0-0d68-406f-aee6-6a4f0eb40703 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:37.258830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e8dd0-0d68-406f-aee6-6a4f0eb40703 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:37.264315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:37.264982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:37.265268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:37.265546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:37.273592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:37.275158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39c289-40e1-455b-85b0-21c5a941336b service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:37.284439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 719/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:38.197847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35871-d8ab-464c-9a6b-ce8c692c74a2 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:38.198384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35871-d8ab-464c-9a6b-ce8c692c74a2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:38.198478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35871-d8ab-464c-9a6b-ce8c692c74a2 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:38.199817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:38.204611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3ae5-4d44-4965-8fc2-9dd5addd9df5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:38.205011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3ae5-4d44-4965-8fc2-9dd5addd9df5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:38.205153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3ae5-4d44-4965-8fc2-9dd5addd9df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:38.223022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9dfa3ae5-4d44-4965-8fc2-9dd5addd9df5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:38.224209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9dfa3ae5-4d44-4965-8fc2-9dd5addd9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:38.232701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:38.384113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35871-d8ab-464c-9a6b-ce8c692c74a2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:38.399090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35871-d8ab-464c-9a6b-ce8c692c74a2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:38.404487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:38.405144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:38.405429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:38.405708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:38.413870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:38.415968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b730a73-189e-4198-b21e-1064fa3e8d28 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:38.425329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 711/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:39.336778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee287-7ec9-401d-a380-932a7a71d1a8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:39.337311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee287-7ec9-401d-a380-932a7a71d1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:39.337311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee287-7ec9-401d-a380-932a7a71d1a8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:39.338583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:21:39.343423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffabb0-756b-4af7-af56-17228706275f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:39.343820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffabb0-756b-4af7-af56-17228706275f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:39.343992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffabb0-756b-4af7-af56-17228706275f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:39.362956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-faffabb0-756b-4af7-af56-17228706275f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:39.364149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-faffabb0-756b-4af7-af56-17228706275f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:39.373432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:39.521347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee287-7ec9-401d-a380-932a7a71d1a8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:39.535240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee287-7ec9-401d-a380-932a7a71d1a8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:39.541012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:39.541633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:39.541911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:39.542184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:39.550119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:39.551682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a123f-3f38-46f5-861f-118464b4a2f9 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:39.561431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 722/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:40.481280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013fcc-db91-4bb8-9100-ecc0f050df0e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:40.481859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013fcc-db91-4bb8-9100-ecc0f050df0e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:40.481859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013fcc-db91-4bb8-9100-ecc0f050df0e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:40.483193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:40.488219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec9e8-ea19-47d0-8208-572819ebdec6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:40.488675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec9e8-ea19-47d0-8208-572819ebdec6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:40.488812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec9e8-ea19-47d0-8208-572819ebdec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:40.506868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-213ec9e8-ea19-47d0-8208-572819ebdec6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:40.508699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-213ec9e8-ea19-47d0-8208-572819ebdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:40.516253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:21:40.668207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013fcc-db91-4bb8-9100-ecc0f050df0e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:40.682897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013fcc-db91-4bb8-9100-ecc0f050df0e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:40.688237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:40.688975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:40.689257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:40.689534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:40.698272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:40.699837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a579b-5f4d-46f8-96a4-13261bd54b41 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:40.709204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 714/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:41.584029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef0e5-1633-49ff-966c-e09944228245 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:41.584575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef0e5-1633-49ff-966c-e09944228245 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:41.584726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef0e5-1633-49ff-966c-e09944228245 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:41.586649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:41.591642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9514-ed70-4140-bb49-01a26d9c11b0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:41.592039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9514-ed70-4140-bb49-01a26d9c11b0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:41.592201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9514-ed70-4140-bb49-01a26d9c11b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:41.609767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1bfc9514-ed70-4140-bb49-01a26d9c11b0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:41.610948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1bfc9514-ed70-4140-bb49-01a26d9c11b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:41.618967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:41.767807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef0e5-1633-49ff-966c-e09944228245 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:41.782880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef0e5-1633-49ff-966c-e09944228245 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:41.788251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:41.788966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:41.789252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:41.789530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:41.797580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:41.799152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743ec85-846c-4062-b1a0-5b2b9d417ee3 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:41.808533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 725/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:43.931916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753872b0-a5b7-46e0-92d9-be6502a86fea None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:43.932468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753872b0-a5b7-46e0-92d9-be6502a86fea None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:43.932468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753872b0-a5b7-46e0-92d9-be6502a86fea None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:43.933900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:43.939199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973467ab-b891-417e-86c4-b9bc6f269a75 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:43.939595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973467ab-b891-417e-86c4-b9bc6f269a75 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:43.939750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973467ab-b891-417e-86c4-b9bc6f269a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:43.958311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-973467ab-b891-417e-86c4-b9bc6f269a75 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:43.959494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-973467ab-b891-417e-86c4-b9bc6f269a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:43.967859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:44.005677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753872b0-a5b7-46e0-92d9-be6502a86fea None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:44.017029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753872b0-a5b7-46e0-92d9-be6502a86fea None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:44.020555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:44.021211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:44.021494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:44.021768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:44.031661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:44.033139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1996c5-400a-4217-9f90-ec7940f8bfc6 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:44.042459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 717/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:21:46.459214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d0a2-d489-4fd0-b75c-bcd29839c6ae None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:46.459751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d0a2-d489-4fd0-b75c-bcd29839c6ae None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:46.459751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496d0a2-d489-4fd0-b75c-bcd29839c6ae None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:46.461782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:46.467175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dac47-2487-4ac4-baa6-7a3a3ecc7daa None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:46.467589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dac47-2487-4ac4-baa6-7a3a3ecc7daa None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:46.467727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dac47-2487-4ac4-baa6-7a3a3ecc7daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:46.486077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-423dac47-2487-4ac4-baa6-7a3a3ecc7daa None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:46.487265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-423dac47-2487-4ac4-baa6-7a3a3ecc7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:46.499973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:46.537794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496d0a2-d489-4fd0-b75c-bcd29839c6ae None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:46.548779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496d0a2-d489-4fd0-b75c-bcd29839c6ae None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:46.552384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:46.553055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:46.553337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:46.553614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:46.563139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:46.564567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434426-b21b-467a-b367-31f34dec51c3 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:46.574513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 728/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:49.170251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2587-0ae4-47f7-9ab2-f5de4a87b00f None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:49.170864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2587-0ae4-47f7-9ab2-f5de4a87b00f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:49.170864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2587-0ae4-47f7-9ab2-f5de4a87b00f None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:49.172180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:21:49.177570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0698-6ad5-47f4-b889-fdd7c28958f8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:49.178017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0698-6ad5-47f4-b889-fdd7c28958f8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:49.178110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0698-6ad5-47f4-b889-fdd7c28958f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:49.196190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a84c0698-6ad5-47f4-b889-fdd7c28958f8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:49.197497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a84c0698-6ad5-47f4-b889-fdd7c28958f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:49.205251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:49.244161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf2587-0ae4-47f7-9ab2-f5de4a87b00f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:49.254638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf2587-0ae4-47f7-9ab2-f5de4a87b00f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:49.258162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:49.258778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:49.259058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:49.259328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:49.268004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:49.269442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1edc0-0ea0-4e62-bb29-f4dbf427c03b service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:49.278935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 720/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:51.646761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:51.647333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:51.647333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:51.649201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:51.654490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d615ae-cbb9-4e83-9bad-a9e96da746b6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:51.654926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d615ae-cbb9-4e83-9bad-a9e96da746b6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:51.655040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d615ae-cbb9-4e83-9bad-a9e96da746b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:51.673814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e3d615ae-cbb9-4e83-9bad-a9e96da746b6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:51.675010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e3d615ae-cbb9-4e83-9bad-a9e96da746b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:51.682815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:51.721792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:51.723521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:51.736663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b9118-7701-4db6-ae5e-bbc3730724ef None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:51.740114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:51.740774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:51.741062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:51.741340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:51.750946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:51.752369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b455aa7-875d-43d1-b059-1529529b72b2 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:51.762265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 731/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:54.122600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207e182-23b2-42b7-89f6-89e4733e4d33 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:54.123157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207e182-23b2-42b7-89f6-89e4733e4d33 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:54.123157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207e182-23b2-42b7-89f6-89e4733e4d33 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:54.124555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:54.129865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25133-e58c-4448-9dee-282e7d8487ca None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:54.130300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25133-e58c-4448-9dee-282e7d8487ca None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:54.130434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25133-e58c-4448-9dee-282e7d8487ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:54.149000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3fc25133-e58c-4448-9dee-282e7d8487ca None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:54.150197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fc25133-e58c-4448-9dee-282e7d8487ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:54.157939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:54.195331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207e182-23b2-42b7-89f6-89e4733e4d33 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:54.206011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207e182-23b2-42b7-89f6-89e4733e4d33 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:54.209463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:54.210603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:54.210879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:54.211149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:54.219868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:54.221316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef98079-096d-4eaf-9b38-a8c438a9c5d1 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:54.231200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 723/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:56.511075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474654a-929f-4475-b4f8-98e5e351b4bf None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:56.511615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474654a-929f-4475-b4f8-98e5e351b4bf None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:56.511615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474654a-929f-4475-b4f8-98e5e351b4bf None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:56.513023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:56.518328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3e1e1-4c17-48ba-a6d0-6c95fceafa62 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:56.518754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3e1e1-4c17-48ba-a6d0-6c95fceafa62 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:56.518885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3e1e1-4c17-48ba-a6d0-6c95fceafa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:56.536465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9fb3e1e1-4c17-48ba-a6d0-6c95fceafa62 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:56.537677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9fb3e1e1-4c17-48ba-a6d0-6c95fceafa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:56.545897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:56.583489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474654a-929f-4475-b4f8-98e5e351b4bf None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:56.594242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474654a-929f-4475-b4f8-98e5e351b4bf None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:56.597746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:56.598378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:56.598661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:56.598936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:56.608228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:56.609705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883b08f-abc8-4156-9a2d-ba9ed0f120a3 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:56.619883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 734/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:58.879934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58edfe5-3165-4130-ae9e-fe3c6c1b4d9c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:58.880475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58edfe5-3165-4130-ae9e-fe3c6c1b4d9c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:58.880475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58edfe5-3165-4130-ae9e-fe3c6c1b4d9c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:58.882430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:21:58.887631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01526af1-1fe8-4f95-9645-ab0e153ac69d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:58.888037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01526af1-1fe8-4f95-9645-ab0e153ac69d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:58.888218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01526af1-1fe8-4f95-9645-ab0e153ac69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:58.894198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=70 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:58.906603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-01526af1-1fe8-4f95-9645-ab0e153ac69d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:58.907786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-01526af1-1fe8-4f95-9645-ab0e153ac69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:58.915535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:58.952933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58edfe5-3165-4130-ae9e-fe3c6c1b4d9c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:58.958414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=73 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:58.964880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58edfe5-3165-4130-ae9e-fe3c6c1b4d9c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:58.968361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:58.969059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:58.969333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:58.969605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:58.978683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:58.980182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea88a8d-40ac-4f6d-bcc3-d78775d1ec3f service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:58.990166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 726/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:01.244258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf2df-686a-43b8-9da7-52071a8d6bc8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:01.244824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf2df-686a-43b8-9da7-52071a8d6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:01.244824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf2df-686a-43b8-9da7-52071a8d6bc8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:01.246329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:01.251675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827cab5-e3e9-402d-8d8b-c502733eb8b7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:01.252105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827cab5-e3e9-402d-8d8b-c502733eb8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:01.252220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827cab5-e3e9-402d-8d8b-c502733eb8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:01.270197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2827cab5-e3e9-402d-8d8b-c502733eb8b7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:01.271323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2827cab5-e3e9-402d-8d8b-c502733eb8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:01.279678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:01.316076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccf2df-686a-43b8-9da7-52071a8d6bc8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:01.327368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccf2df-686a-43b8-9da7-52071a8d6bc8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:01.330939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:01.331560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:01.331843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:01.332114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:01.341720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:01.343152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb0eb6-eb49-4aa8-ba54-8e7c0e209aa0 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:01.352337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 737/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:03.579098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8721-5da8-4529-9f97-ae9d05c1823a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:03.579666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8721-5da8-4529-9f97-ae9d05c1823a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:03.579666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe8721-5da8-4529-9f97-ae9d05c1823a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:03.581068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:03.586214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ceee-7a7b-47d4-a4a4-222d100bdc75 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:03.586643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ceee-7a7b-47d4-a4a4-222d100bdc75 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:03.586748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ceee-7a7b-47d4-a4a4-222d100bdc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:03.604558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3ae8ceee-7a7b-47d4-a4a4-222d100bdc75 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:03.605760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3ae8ceee-7a7b-47d4-a4a4-222d100bdc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:03.614007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:03.651951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe8721-5da8-4529-9f97-ae9d05c1823a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:03.662461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe8721-5da8-4529-9f97-ae9d05c1823a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:03.666577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:03.667201 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:03.667481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:03.667754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:03.676501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:03.677935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f4238-90b0-4529-a259-41e95b39b020 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:03.687202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 729/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:05.899974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6cc15-b8ef-4585-bcbb-35fb40b6a9f4 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:05.909666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6cc15-b8ef-4585-bcbb-35fb40b6a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:05.909666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6cc15-b8ef-4585-bcbb-35fb40b6a9f4 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:05.909666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:05.909984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36642f9-bd85-40fd-9f0d-c94733e3ae54 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:05.909984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36642f9-bd85-40fd-9f0d-c94733e3ae54 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:05.909984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36642f9-bd85-40fd-9f0d-c94733e3ae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:05.925115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d36642f9-bd85-40fd-9f0d-c94733e3ae54 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:05.926287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d36642f9-bd85-40fd-9f0d-c94733e3ae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:05.934981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:05.970853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6cc15-b8ef-4585-bcbb-35fb40b6a9f4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:05.981731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6cc15-b8ef-4585-bcbb-35fb40b6a9f4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:05.985304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:05.985984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:05.986292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:05.986565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:05.996214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:05.997773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdbeee-eb00-4de9-a0ab-2806fb8492d1 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:06.007898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 740/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:15.920353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=17 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:22:16.003026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=17 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:22:38.616016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748ac4-5c50-46df-8a85-c08b91080ee4 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:38.616538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748ac4-5c50-46df-8a85-c08b91080ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:38.616668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748ac4-5c50-46df-8a85-c08b91080ee4 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:38.618270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 731/1471] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:38.623460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55775c9-754d-4f2d-832e-1533ae83e3ae None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:38.623948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55775c9-754d-4f2d-832e-1533ae83e3ae None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:38.624055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55775c9-754d-4f2d-832e-1533ae83e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:38.641076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e55775c9-754d-4f2d-832e-1533ae83e3ae None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:38.642249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e55775c9-754d-4f2d-832e-1533ae83e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:38.649898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 741/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:38.696326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2929-e452-4fb9-8855-1e8af462db62 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:38.696646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2929-e452-4fb9-8855-1e8af462db62 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:38.696930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2929-e452-4fb9-8855-1e8af462db62 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:38.698284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 732/1473] 10.222.0.57 () {54 vars in 962 bytes} [Tue Jun 7 09:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:38.703126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16add47-065c-49f6-8894-87c2fe7e6ce2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:38.703411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16add47-065c-49f6-8894-87c2fe7e6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:38.703687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16add47-065c-49f6-8894-87c2fe7e6ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:38.721006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a16add47-065c-49f6-8894-87c2fe7e6ce2 None None] MFA Rules not processed for user `7025724d5c31460db223411aab77003a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:38.722177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a16add47-065c-49f6-8894-87c2fe7e6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:38.729050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 742/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Jun 7 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:38.736031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a2929-e452-4fb9-8855-1e8af462db62 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:38.747311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a2929-e452-4fb9-8855-1e8af462db62 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:38.750793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:38.751435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:38.751712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:38.751980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:38.760734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:38.762138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eddd97-4955-4749-9fb3-fbfe8ea5faab service glance] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:38.771305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 733/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:22:41.081836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dbe5-d962-4a67-aea4-d468b06fcc5c None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:41.082376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dbe5-d962-4a67-aea4-d468b06fcc5c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:41.082376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dbe5-d962-4a67-aea4-d468b06fcc5c None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:41.084418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 743/1476] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:41.089612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962aceb-3e74-4286-8e96-cd6c1c85dc68 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:41.090073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962aceb-3e74-4286-8e96-cd6c1c85dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:41.090183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962aceb-3e74-4286-8e96-cd6c1c85dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:41.107642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b962aceb-3e74-4286-8e96-cd6c1c85dc68 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:41.108863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b962aceb-3e74-4286-8e96-cd6c1c85dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:41.116493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 734/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:41.140908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dbe5-d962-4a67-aea4-d468b06fcc5c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:41.154085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dbe5-d962-4a67-aea4-d468b06fcc5c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:41.158612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:41.159251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:41.159534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:41.159807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:41.160535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:41.162347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:22:41.162347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:22:41.162347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:22:41.163491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aa45e-7654-40e3-9288-c350d625a4e5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:41.182849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 744/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Jun 7 09:22:41 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:22:43.422454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2466b-8bfa-40f4-a922-82bb84008065 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:43.423023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2466b-8bfa-40f4-a922-82bb84008065 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:43.423023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2466b-8bfa-40f4-a922-82bb84008065 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:43.424376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 735/1479] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:43.430301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e709-2cb7-4c0b-9f89-d400d36d6cf8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:43.430712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e709-2cb7-4c0b-9f89-d400d36d6cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:43.430854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e709-2cb7-4c0b-9f89-d400d36d6cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:43.448970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e872e709-2cb7-4c0b-9f89-d400d36d6cf8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:43.450148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e872e709-2cb7-4c0b-9f89-d400d36d6cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:43.457805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 745/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:43.502247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2466b-8bfa-40f4-a922-82bb84008065 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:43.517354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2466b-8bfa-40f4-a922-82bb84008065 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:43.522876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:43.523535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:43.523815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:43.524117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:43.532469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:43.534123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0207b93-2f4e-4759-8837-8c4bd9b7f0c1 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:43.543443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 736/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:45.923627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5579-3d4b-43cf-88cb-e8cce20949bb None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:45.924173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5579-3d4b-43cf-88cb-e8cce20949bb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:45.924173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5579-3d4b-43cf-88cb-e8cce20949bb None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:45.925561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 746/1482] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:45.930945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d9162-b690-49af-9073-0cebc5e56c2f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:45.931357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d9162-b690-49af-9073-0cebc5e56c2f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:45.931511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d9162-b690-49af-9073-0cebc5e56c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:45.949378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-573d9162-b690-49af-9073-0cebc5e56c2f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:45.951119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-573d9162-b690-49af-9073-0cebc5e56c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:45.959255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 737/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:45.996225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5579-3d4b-43cf-88cb-e8cce20949bb None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:46.007308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5579-3d4b-43cf-88cb-e8cce20949bb None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:46.010824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:46.011454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:46.011733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:46.012004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:46.021694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:46.023122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39957fff-5932-4142-af7c-ae5285f0a2ab service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:46.032399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 747/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:48.293375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2accd-a685-4637-9df8-53f48cbb1bba None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:48.293915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2accd-a685-4637-9df8-53f48cbb1bba None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:48.293915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2accd-a685-4637-9df8-53f48cbb1bba None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:48.295219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 738/1485] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:48.300466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9090f9-a2ad-430d-9d6b-4abea9df58e2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:48.300954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9090f9-a2ad-430d-9d6b-4abea9df58e2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:48.301057 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9090f9-a2ad-430d-9d6b-4abea9df58e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:48.318990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bd9090f9-a2ad-430d-9d6b-4abea9df58e2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:48.320188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bd9090f9-a2ad-430d-9d6b-4abea9df58e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:48.327952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 748/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:48.365107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2accd-a685-4637-9df8-53f48cbb1bba None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:48.376004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2accd-a685-4637-9df8-53f48cbb1bba None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:48.379537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:48.380710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:48.380995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:48.381269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:48.390199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:48.391613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfacbdf-a144-441c-8145-f7c4dea40349 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:48.401559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 739/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:50.792993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d69361-3a55-477e-a37d-b6b103fd14f9 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:50.793619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d69361-3a55-477e-a37d-b6b103fd14f9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:50.793619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d69361-3a55-477e-a37d-b6b103fd14f9 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:50.794990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 749/1488] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:50.800338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9baa-6815-47dd-96f9-80558da9dc53 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:50.800830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9baa-6815-47dd-96f9-80558da9dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:50.801004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9baa-6815-47dd-96f9-80558da9dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:50.819200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e10e9baa-6815-47dd-96f9-80558da9dc53 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:50.820408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e10e9baa-6815-47dd-96f9-80558da9dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:50.828088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 740/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:50.868106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d69361-3a55-477e-a37d-b6b103fd14f9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:50.878837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d69361-3a55-477e-a37d-b6b103fd14f9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:50.882410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:50.883033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:50.883314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:50.883588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:50.892612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:50.896296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde135a-eef8-4db6-8e3d-427fbcd6a33d service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:50.903750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 750/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:53.576947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838382-6c3a-441e-9ae7-05dfec1a6e5c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:53.577514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838382-6c3a-441e-9ae7-05dfec1a6e5c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:53.577514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838382-6c3a-441e-9ae7-05dfec1a6e5c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:53.578891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 741/1491] 10.222.0.57 () {54 vars in 926 bytes} [Tue Jun 7 09:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:22:53.584113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce807f-0c9c-49d9-925e-021886808bfc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:53.584562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce807f-0c9c-49d9-925e-021886808bfc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:53.584708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce807f-0c9c-49d9-925e-021886808bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:53.602627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-71ce807f-0c9c-49d9-925e-021886808bfc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:22:53.603829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-71ce807f-0c9c-49d9-925e-021886808bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:22:53.611553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 751/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Tue Jun 7 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:22:53.649939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51838382-6c3a-441e-9ae7-05dfec1a6e5c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:53.660455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51838382-6c3a-441e-9ae7-05dfec1a6e5c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:53.664386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:53.665038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:53.665316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:53.665585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:53.674166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:53.675572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b862eb-cdba-4b45-b2fc-56496428ecd2 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:53.685322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 742/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:03.597027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=33 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:03.681247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=33 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:24:36.955114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:36.955719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:36.955719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:36.973887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:24:36.975062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:36.982701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 752/1494] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:24:36.990441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:36.990928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:36.991033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.008893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:24:37.010070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:37.018399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 743/1495] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:24:37.022811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.032601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bb811-3bb4-4b67-8f9c-fc5926bad4b9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.037180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.037822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.038101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.038374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.039091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:24:37.039393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.040396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.040396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:24:37.040396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:24:37.041330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.052675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 753/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:24:37.060486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.069985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb5bcd-927f-4cdc-98e8-9ddb4bc89d87 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.074413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.075046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.075324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.075595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.076282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.077991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.077991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:24:37.077991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:24:37.079165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.208288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 744/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:24:37 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:24:37.212262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.222532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5278c-f347-407c-88d8-78ea91939bdd admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.278566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.279221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.279500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.279772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.280458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.281491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.281491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:24:37.281491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:24:37.282701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.384224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 754/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:24:37 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:24:37.388555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.399129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0caf3-be01-4262-bbae-db3730405fb6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.403637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.404296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.404599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.404887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.405635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.406634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.406634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:24:37.406634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:24:37.407512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.420214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 745/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:24:37.424726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.434722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293803f2-46d7-49c2-95fb-f06ea9bb2c19 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.439918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:24:37.444868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:24:37.445470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a72dc-7e3f-4092-a07e-5afb9deb406d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.457496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 755/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:24:37.461704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.471918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486425-330e-49b8-89cc-bad706918158 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.476504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.477420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.477700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.477974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] PATH_INFO: `/v3/projects/5a69c1ea901a4600a19fa58dc51212eb/users/9188175b94af4e93abc455d05cc428e4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.493869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a69c1ea901a4600a19fa58dc51212eb, user_id=9188175b94af4e93abc455d05cc428e4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.494925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.494925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:24:37.494925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:24:37.496864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45756d-15fd-4323-9421-f05c96f7fbe5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.583726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 746/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:24:37 2022] PUT /identity/v3/projects/5a69c1ea901a4600a19fa58dc51212eb/users/9188175b94af4e93abc455d05cc428e4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:24:37.601205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c502-0ab0-4157-915e-bd783897b22c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.601497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c502-0ab0-4157-915e-bd783897b22c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.601776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c502-0ab0-4157-915e-bd783897b22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.629774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2028c502-0ab0-4157-915e-bd783897b22c None None] MFA Rules not processed for user `9188175b94af4e93abc455d05cc428e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:24:37.630982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2028c502-0ab0-4157-915e-bd783897b22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:37.692472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 756/1502] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:24:37.706036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070b07-c81b-46c8-8c3d-d456dd58e818 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.706327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070b07-c81b-46c8-8c3d-d456dd58e818 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.706606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070b07-c81b-46c8-8c3d-d456dd58e818 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.707978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 747/1503] 10.222.0.57 () {54 vars in 963 bytes} [Tue Jun 7 09:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:24:37.712886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e794f1-590f-4025-966b-7474683668bc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.713119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e794f1-590f-4025-966b-7474683668bc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.713394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e794f1-590f-4025-966b-7474683668bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.731185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d5e794f1-590f-4025-966b-7474683668bc None None] MFA Rules not processed for user `c7d851c1cb8b495a87692b8e8970f851`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:24:37.732359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d5e794f1-590f-4025-966b-7474683668bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:37.774692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:24:37.781682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 757/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Tue Jun 7 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:24:37.789928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070b07-c81b-46c8-8c3d-d456dd58e818 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.850301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070b07-c81b-46c8-8c3d-d456dd58e818 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.850301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.850301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.850301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.851101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.851101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.851101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:37.851101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 748/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:24:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:24:37.883248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c867fd-50a8-42e1-8b81-efe0be2baccf None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.883541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c867fd-50a8-42e1-8b81-efe0be2baccf None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.883824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c867fd-50a8-42e1-8b81-efe0be2baccf None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.885153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 758/1506] 10.222.0.57 () {52 vars in 829 bytes} [Tue Jun 7 09:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:24:37.888674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:37.951356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904413-9171-43bf-b99e-04eac1145293 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:37.955901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:37.956800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:37.957089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:37.957369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] PATH_INFO: `/v3/users/9188175b94af4e93abc455d05cc428e4` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:37.958553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9188175b94af4e93abc455d05cc428e4)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:37.959652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:37.959652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:24:37.959652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:24:37.960885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:38.150023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9188175b94af4e93abc455d05cc428e4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:38.163049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9188175b94af4e93abc455d05cc428e4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:38.217274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9188175b94af4e93abc455d05cc428e4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:38.218004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9188175b94af4e93abc455d05cc428e4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:38.218618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9188175b94af4e93abc455d05cc428e4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:38.235409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:24:38.244156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 749/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:24:37 2022] DELETE /identity/v3/users/9188175b94af4e93abc455d05cc428e4 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:24:38.248688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:38.249146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:38.249271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:38.267054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:24:38.268210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:38.325834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 759/1508] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:24:38.335314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:38.339033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:38.393716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdfef9-695c-4845-84d6-1b237ad41fbb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:38.397285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:38.397917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:38.398195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:38.398465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:38.407220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:38.408655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e94cc7-c478-4836-8685-ee3dacde6d9f service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:38.417929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 750/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:24:38.933911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:24:38.935712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:24:38.947595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e0c89-1cf4-459b-b4b1-a54dc527369b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:24:38.952427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:24:38.953313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:24:38.953697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:24:38.953900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] PATH_INFO: `/v3/projects/5a69c1ea901a4600a19fa58dc51212eb` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:24:38.955590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a69c1ea901a4600a19fa58dc51212eb)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:24:38.956663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:24:38.956663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:24:38.956663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:24:38.957853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:24:39.108983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a69c1ea901a4600a19fa58dc51212eb', 'request_id': 'req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:39.115349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a69c1ea901a4600a19fa58dc51212eb', 'request_id': 'req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:39.128356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a69c1ea901a4600a19fa58dc51212eb', 'request_id': 'req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:39.211392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] The token cache is being invalidate because project 5a69c1ea901a4600a19fa58dc51212eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:24:39.211590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:39.212311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b95ab29d-bd37-40fd-bd1a-c9240d6de3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:24:39.213889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 760/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:24:38 2022] DELETE /identity/v3/projects/5a69c1ea901a4600a19fa58dc51212eb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:24:48.413813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:24:49.208229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:26:18.541282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:18.541872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:18.541872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:18.543223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:18.548937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d4492-0def-4473-81c7-d647e891acbb None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:18.549414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d4492-0def-4473-81c7-d647e891acbb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:18.549521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d4492-0def-4473-81c7-d647e891acbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:18.567888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-956d4492-0def-4473-81c7-d647e891acbb None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:18.569112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-956d4492-0def-4473-81c7-d647e891acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:18.625871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:18.794817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:18.796451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:18.872259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15385621-1cac-4857-b3b6-9a68802af549 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:18.878006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:18.878646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:18.879043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:18.879323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:18.887544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:18.889231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8a522-ac7a-46c2-ac9a-55209c82fe7c service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:18.898417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 752/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:19.774111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71f5d-d9e3-49b7-a986-cfb86c9f2d15 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:19.774710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71f5d-d9e3-49b7-a986-cfb86c9f2d15 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:19.774710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71f5d-d9e3-49b7-a986-cfb86c9f2d15 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:19.776075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:19.780849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697de88-eaa0-4167-9043-88a292f92258 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:19.781261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697de88-eaa0-4167-9043-88a292f92258 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:19.781401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697de88-eaa0-4167-9043-88a292f92258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:19.799992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7697de88-eaa0-4167-9043-88a292f92258 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:19.801234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7697de88-eaa0-4167-9043-88a292f92258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:19.809061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:19.958786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a71f5d-d9e3-49b7-a986-cfb86c9f2d15 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:19.973501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a71f5d-d9e3-49b7-a986-cfb86c9f2d15 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:19.979431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:19.980075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:19.980357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:19.980660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:19.990129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:19.991715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe543-3378-4731-b48f-514ad6bc33bc service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:20.001085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 763/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:20.008383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36888dc-d9d0-4410-b81c-03d3946d9b69 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:20.008383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36888dc-d9d0-4410-b81c-03d3946d9b69 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:20.008383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36888dc-d9d0-4410-b81c-03d3946d9b69 None None] PATH_INFO: `/v3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:20.009741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 754/1517] 10.222.0.55 () {54 vars in 925 bytes} [Tue Jun 7 09:26:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:26:20.016816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9a26-5ffd-4daf-8c47-54ff87fcbb41 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:20.017367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9a26-5ffd-4daf-8c47-54ff87fcbb41 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:20.017760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9a26-5ffd-4daf-8c47-54ff87fcbb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:20.037471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-319b9a26-5ffd-4daf-8c47-54ff87fcbb41 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:20.038715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-319b9a26-5ffd-4daf-8c47-54ff87fcbb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:20.105907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 764/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Jun 7 09:26:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:26:29.308170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fb8ee-b5b0-4e21-aa10-49cdee57e129 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:29.308774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fb8ee-b5b0-4e21-aa10-49cdee57e129 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:29.308774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fb8ee-b5b0-4e21-aa10-49cdee57e129 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:29.310039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 755/1519] 10.222.0.57 () {54 vars in 960 bytes} [Tue Jun 7 09:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:29.314954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d921-bfaf-4d28-ab73-0188931f725b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:29.315376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d921-bfaf-4d28-ab73-0188931f725b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:29.315507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d921-bfaf-4d28-ab73-0188931f725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:29.333346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2656d921-bfaf-4d28-ab73-0188931f725b None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:29.334550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2656d921-bfaf-4d28-ab73-0188931f725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:29.342910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 765/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Tue Jun 7 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:29.350452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fb8ee-b5b0-4e21-aa10-49cdee57e129 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:29.364388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fb8ee-b5b0-4e21-aa10-49cdee57e129 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:29.369297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:29.369914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:29.370199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:29.370473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:29.378665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:29.380329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65e63-9861-4424-b813-06872fb43a74 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:29.389884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 756/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:31.294588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c929c2-13a4-401f-9320-5991d95c8070 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:31.295152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c929c2-13a4-401f-9320-5991d95c8070 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:31.295152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c929c2-13a4-401f-9320-5991d95c8070 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:31.296432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:31.301780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ffa7-b54b-48b4-8ff0-4d6c19613805 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:31.302195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ffa7-b54b-48b4-8ff0-4d6c19613805 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:31.302343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ffa7-b54b-48b4-8ff0-4d6c19613805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:31.320655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7927ffa7-b54b-48b4-8ff0-4d6c19613805 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:31.321847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7927ffa7-b54b-48b4-8ff0-4d6c19613805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:31.329661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:31.481398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c929c2-13a4-401f-9320-5991d95c8070 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:31.496412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c929c2-13a4-401f-9320-5991d95c8070 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:31.501934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:31.502557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:31.502839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:31.503114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:31.511149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:31.512769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df4c9c-38d9-4736-a907-2865c12cf94d service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:31.522291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 767/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:32.379702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54e478-94b0-48e4-9c64-90c5a00c6a16 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:32.380245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54e478-94b0-48e4-9c64-90c5a00c6a16 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:32.380245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54e478-94b0-48e4-9c64-90c5a00c6a16 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:32.381592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 09:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:32.386207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65942-5648-45a2-ad14-f2c9ba5ededc None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:32.386625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65942-5648-45a2-ad14-f2c9ba5ededc None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:32.386750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65942-5648-45a2-ad14-f2c9ba5ededc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:32.404228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ebf65942-5648-45a2-ad14-f2c9ba5ededc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:32.405447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ebf65942-5648-45a2-ad14-f2c9ba5ededc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:32.413649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 09:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:32.565782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54e478-94b0-48e4-9c64-90c5a00c6a16 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:32.580409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54e478-94b0-48e4-9c64-90c5a00c6a16 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:32.585733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:32.586364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:32.586645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:32.586917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:32.595502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:32.597102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ad6e6-9d61-40d8-8b72-dc72c6ee75e7 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:32.606290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 759/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:26:36.583064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:36.583659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:36.583659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:36.585029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:36.590300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0717-eaec-46c3-914b-42195ecabfda None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:36.590727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0717-eaec-46c3-914b-42195ecabfda None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:36.590831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0717-eaec-46c3-914b-42195ecabfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:36.608406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e11b0717-eaec-46c3-914b-42195ecabfda None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:36.609605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e11b0717-eaec-46c3-914b-42195ecabfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:36.617568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:36.653997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:36.655649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:36.711964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e952f7f-6d49-4027-92c8-bc6c256864bb None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:36.715615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:36.716256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:36.716538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:36.716848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:36.725987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:36.727410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183eb12-abf4-4ec5-9c27-cd985bbe1571 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:36.732317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:26:36.737360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 770/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:38.945013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8173b-6ab2-4fb7-be44-605264feac4a None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:38.945613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8173b-6ab2-4fb7-be44-605264feac4a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:38.945613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8173b-6ab2-4fb7-be44-605264feac4a None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:38.947027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:38.952248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af14e-1ae8-4d1d-aab9-a93b864b4aa4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:38.952713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af14e-1ae8-4d1d-aab9-a93b864b4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:38.952827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af14e-1ae8-4d1d-aab9-a93b864b4aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:38.971648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-180af14e-1ae8-4d1d-aab9-a93b864b4aa4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:38.972804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-180af14e-1ae8-4d1d-aab9-a93b864b4aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:38.980908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:39.017504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8173b-6ab2-4fb7-be44-605264feac4a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:39.028455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8173b-6ab2-4fb7-be44-605264feac4a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:39.032114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:39.033352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:39.033636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:39.033907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:39.042953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:39.044360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a684b-aadc-4bf8-8f63-274936ff3306 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:39.054439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 762/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:42.732795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:42.733247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:42.733349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:42.734708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:42.740827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837b1b9-1f54-4558-9e65-2f337e4b46e7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:42.741294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837b1b9-1f54-4558-9e65-2f337e4b46e7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:42.741393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837b1b9-1f54-4558-9e65-2f337e4b46e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:42.759224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0837b1b9-1f54-4558-9e65-2f337e4b46e7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:42.760528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0837b1b9-1f54-4558-9e65-2f337e4b46e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:42.768286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:43.037055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:43.038726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:43.093880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a8c56-c74d-404f-8c3a-fd055ebc60e5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:43.097958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:43.102336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:43.102336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:43.102336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:43.108150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:43.109598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac381b-d5e6-49de-8b14-73399568c1a6 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:43.119101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 773/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:45.359696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:45.360260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:45.360260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:45.362379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 09:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:26:45.367753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154358-d284-4fb2-bd1a-803d1d3a40c7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:45.368162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154358-d284-4fb2-bd1a-803d1d3a40c7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:45.368283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154358-d284-4fb2-bd1a-803d1d3a40c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:45.386605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d0154358-d284-4fb2-bd1a-803d1d3a40c7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:26:45.387834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d0154358-d284-4fb2-bd1a-803d1d3a40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:45.395537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 09:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:26:45.663425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:45.666035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:45.679219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447f8f-43e5-43af-b6da-ede165fdb2bc None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:45.682795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:45.683422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:45.683705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:45.683982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:45.693044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:45.694455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b5623-1719-487a-92a6-dcb2be0cc6d1 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:45.703869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 765/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:26:55.381485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=32 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:26:55.699822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=70 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:17.035788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b17-a749-4938-824e-1bc7cb3ada38 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.036397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b17-a749-4938-824e-1bc7cb3ada38 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.036397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b17-a749-4938-824e-1bc7cb3ada38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.060213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89ec5-9390-453e-b365-b9d8a99ee4dd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.061839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89ec5-9390-453e-b365-b9d8a99ee4dd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.062363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89ec5-9390-453e-b365-b9d8a99ee4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.068596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-69622b17-a749-4938-824e-1bc7cb3ada38 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.070166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-69622b17-a749-4938-824e-1bc7cb3ada38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.085563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 775/1540] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.094117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-80a89ec5-9390-453e-b365-b9d8a99ee4dd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.094561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fceab-c4b6-4d7f-ad69-e683cf6c7816 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.094852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fceab-c4b6-4d7f-ad69-e683cf6c7816 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.095644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fceab-c4b6-4d7f-ad69-e683cf6c7816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.096390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-80a89ec5-9390-453e-b365-b9d8a99ee4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.112545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 766/1541] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.121769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.122383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.122889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.126157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-c71fceab-c4b6-4d7f-ad69-e683cf6c7816 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.127516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c71fceab-c4b6-4d7f-ad69-e683cf6c7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.139533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 776/1542] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:17.141015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.141304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.141580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.167164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.167614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.168874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.169235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.181937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 777/1543] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.183586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.192607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 767/1544] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:17.200051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ec0f-7b6c-47e0-a8a4-4b8a48f8e254 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.206065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.206734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.207016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.207292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.207628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.208024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.208329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.209956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6485dd2-8828-41f2-a64b-327b125877d7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.224174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a9f74-5832-4317-8c93-8d77fda14a67 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.232613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 778/1545] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.235446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.235738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.236011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.239442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.240103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.240379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.240669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.241596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.241904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.242910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:17.242910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:28:17.242910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:28:17.243790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a341de-2d66-471e-aa2a-2aa3bd906d08 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.269013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.270283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.271890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 768/1546] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.273672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053f23-3154-4198-a1f0-8955d272621b None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.274267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053f23-3154-4198-a1f0-8955d272621b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.274773 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053f23-3154-4198-a1f0-8955d272621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.282539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 779/1547] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.285882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.303596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9930ce-4519-4db0-a0f1-9df5a4b6735e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.307383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-36053f23-3154-4198-a1f0-8955d272621b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.308284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.308930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-36053f23-3154-4198-a1f0-8955d272621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.309072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.309371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.309654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.310466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.311680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:17.311680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:28:17.311680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:28:17.312888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.317378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 769/1548] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.318618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c615c82-d4d3-401a-9a77-5661fc5e12f5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.318906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c615c82-d4d3-401a-9a77-5661fc5e12f5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.319179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c615c82-d4d3-401a-9a77-5661fc5e12f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.336838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3c615c82-d4d3-401a-9a77-5661fc5e12f5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.338030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3c615c82-d4d3-401a-9a77-5661fc5e12f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.346288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 770/1549] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.349880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.349880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.349880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.366389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.367562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.375338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 771/1550] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.376092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.386390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2325c5-3062-4f57-9cce-3c685d313006 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.391486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.392123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.392413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.392725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.393415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.395298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d3c4-9111-4335-bdfc-2b4a5422edf1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.414700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 780/1551] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:17.417677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.476902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d7313-00ac-4448-b232-b06debe57626 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.481552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.482201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.482486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.482767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.484351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.488165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 772/1552] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:17.488349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.488472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4841-6f18-4517-80d7-1fad97800499 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.488568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4d0c-2db2-40b6-b1d7-3b8d737949a3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.488711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4841-6f18-4517-80d7-1fad97800499 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.488711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4841-6f18-4517-80d7-1fad97800499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.500401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 781/1553] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.501690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a958e-114d-40ef-bcd9-1172ee1e65c6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.501983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a958e-114d-40ef-bcd9-1172ee1e65c6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.502259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a958e-114d-40ef-bcd9-1172ee1e65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.513824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0f1b4841-6f18-4517-80d7-1fad97800499 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.515261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0f1b4841-6f18-4517-80d7-1fad97800499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.520816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-596a958e-114d-40ef-bcd9-1172ee1e65c6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.521994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-596a958e-114d-40ef-bcd9-1172ee1e65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.574868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 773/1554] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.576162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142f350-0b1f-487a-9fa0-e8dc9e8e6049 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.576517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142f350-0b1f-487a-9fa0-e8dc9e8e6049 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.576976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142f350-0b1f-487a-9fa0-e8dc9e8e6049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.584865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 782/1555] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.584865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.584865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.584865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.595411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4142f350-0b1f-487a-9fa0-e8dc9e8e6049 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.596281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4142f350-0b1f-487a-9fa0-e8dc9e8e6049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.604270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 774/1556] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.605370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.605657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.605934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.606212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.607483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.618566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 783/1557] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.619340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.626875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.628049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.629359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481d260-fa75-4916-a0a9-6b81c92a643d None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.633772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.634432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.634728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.634997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.635672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.635892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 775/1558] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.636661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.637681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.645808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429a58e-30c6-4517-aee9-0a0838b52308 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.650406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.651048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.651331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.651609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.652292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.653343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:17.653343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:28:17.653343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:28:17.658188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efa10f-7898-4d87-8923-675c6992c6ab admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.740184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 784/1559] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:17.741178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.751279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d9d4-45e7-4fad-ae1d-ff5c75a8ba7a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.755788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.756409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.756728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.757007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.757684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.759617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.763706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:17.768156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 776/1560] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:17.772699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af471-f19f-4451-bb42-a52a0558e38d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.772699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af471-f19f-4451-bb42-a52a0558e38d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.772699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af471-f19f-4451-bb42-a52a0558e38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.788547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-007af471-f19f-4451-bb42-a52a0558e38d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.789817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-007af471-f19f-4451-bb42-a52a0558e38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.798278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 777/1561] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.806464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.806464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.806464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.817660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:17.818803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:17.826486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 778/1562] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:17.827166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.836047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1074cb-6381-4a09-912f-84634e573c04 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.840655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.841266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.841550 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.841825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.843046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.843349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.844943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.856121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 779/1563] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.856901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.866762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3304f-90ee-46b4-a164-2e8317df2e5c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.871341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.871952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.872231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.872503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.873270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.873593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.875124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.886624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 780/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.887392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.896916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11ee9b-bd6f-4387-b41a-247c28e646c8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.900492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 785/1565] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:17.903939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.904049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.904049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.904049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.904049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.904369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:17.904369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.906205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.918339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 781/1566] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:17.921923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.932914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d030-d4b7-488e-81d2-7dbfc0e15393 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.937643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:17.965381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b779234-7bda-41c6-8c66-bdd503b8f4e8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:17.970829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.971547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.971870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.972149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.972910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.974523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.985548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 786/1567] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:17.986189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:17.990441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:17.991297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:17.993704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:17.993704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:17.993704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:17.994686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:17.996252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91fc82-f947-4098-b741-f8a080cc73fb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.001192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.001828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.002113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.002392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.003117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:18.003451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.005073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.005202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 782/1568] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:18.008396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.018816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23dde-6f20-417a-a55b-30658bf1873a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.021573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 787/1569] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:18.022385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.023976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.024658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.024947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.025224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.025949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:18.026258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.027910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.032991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ff1e-1f2f-4df7-957a-715b8b68db3f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.037631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.038277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.038535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.038809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.039522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.040720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 783/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:18.041479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.041731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.052338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aecb0d-3bae-4f49-8f75-4cd9a4bcd7a6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.056890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.057516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.057794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.058065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.058739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.060651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.163715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 788/1571] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.164462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.178066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 784/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.178066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.226939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d139-fbd0-471d-971b-a757756967a6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.238197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92542bd-a4ed-4bdf-abce-02ca111a1cb9 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.242761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.243415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.243690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.243962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.244667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.246538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.276561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:18.283196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.283865 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.284149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.284424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.285204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.287182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.317889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 785/1573] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.318713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.373507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:18.382456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 789/1574] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.382626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4724-5fd9-4e66-80d6-f3258e2f1d74 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.382727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.384244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.385004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.385271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.385551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.386528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.387926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.389795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765651-5281-47bb-844b-d40846bc8ccb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.395722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.395722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.395722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.395722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.396488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.397952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 786/1575] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:18.398081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.398731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.408150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd57f9-7278-4eee-89e2-e35d7e44c47c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.411884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 790/1576] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:18.411884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.416812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.416812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.416812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.416812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.417215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.417215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.422923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135d7-37d2-430c-98ef-25b92577a59f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.428003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.428312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.428644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.429055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.429724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.431797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.528958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 787/1577] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.530252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.561087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 791/1578] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.561087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.592045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d6ea6-916b-433e-86ed-717f76f16d6b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.621969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74958f-62b7-4e74-961c-fc200cd687e6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.626733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.627403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.627691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.627967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.628679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.630641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.649024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.649689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.649975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.650286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.653557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.653557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.750920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.751853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.762157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62558e37-efe6-4b1c-b4b5-e8b13cf1f7cb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.767277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.767937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.768221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.768496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.769212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.771177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.789357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.790302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.800597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3701f-e9a9-43f8-9d7e-75c224765b2e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.805064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.805687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.805964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.806269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.806979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.808530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.818280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 789/1581] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:18.819003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.827960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbe68d-d7b3-4744-9629-22013b4a49e5 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.832360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.833204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.833487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.833755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] PATH_INFO: `/v3/projects/986f0a9e430e4caaaed4affc2d4b8aeb/users/9726e11bda974054a1e972c097b1ea1d/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.851970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 793/1582] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:18.852158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=986f0a9e430e4caaaed4affc2d4b8aeb, user_id=9726e11bda974054a1e972c097b1ea1d, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.852257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.852366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.862395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dcfcf-3da7-4e86-bf6c-77a3b714a6d8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:18.867310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:18.868172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:18.868459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:18.868772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] PATH_INFO: `/v3/projects/02c52b4e82354dd5af2f6daa818b6753/users/9ea8a3d820c944248e7e4e5c4683e44e/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:18.883368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c52b4e82354dd5af2f6daa818b6753, user_id=9ea8a3d820c944248e7e4e5c4683e44e, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:18.884395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:18.884395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:28:18.884395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:28:18.886467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:18.949106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 790/1583] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:18 2022] PUT /identity/v3/projects/986f0a9e430e4caaaed4affc2d4b8aeb/users/9726e11bda974054a1e972c097b1ea1d/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:18.950151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:18.984343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:19.007379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 794/1584] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:18 2022] PUT /identity/v3/projects/02c52b4e82354dd5af2f6daa818b6753/users/9ea8a3d820c944248e7e4e5c4683e44e/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:19.008729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.014613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbdff-e016-4582-a700-c892d1f5e7bc admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.082017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.082717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.082961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.083288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.083565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc8e9e-5893-4c42-af12-2205132b7e7e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.083949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.085947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.088364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.089059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.089340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.089612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.090291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.092217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.099488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:19.177154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 795/1585] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:19 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:19.177895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 791/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:18 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:19.178860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.181092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.189577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666ae25-7a23-402b-a142-1d7f5eaa5cb2 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.190553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29e8e-cf6e-46f6-88b7-128f2d95564b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.194293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.195025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.195235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.195355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.195566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.195877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.196158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.196321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.196427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.197171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.202350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.202479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.209278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 796/1587] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.213142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.213298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 792/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.213298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.221776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496dfb-cc61-4409-8beb-79681699e0bd admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.222880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8cd1-4192-4506-99e6-dcded4614db0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.226428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.227069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.227351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.227626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.227730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.228369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.228474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.228697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.230651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.230651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.230841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b2ea3-7ae8-42d5-9107-b2883b2b6483 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.231413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf04506-bf54-400e-9610-be3f25223f12 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.241062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 793/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.242282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799984-df92-40e0-84f5-5dedef65518e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.242653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799984-df92-40e0-84f5-5dedef65518e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.242841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799984-df92-40e0-84f5-5dedef65518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.245648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 797/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.246896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.247193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.247472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.260404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-95799984-df92-40e0-84f5-5dedef65518e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:19.261595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-95799984-df92-40e0-84f5-5dedef65518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:19.270052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 794/1591] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:19.270821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95799984-df92-40e0-84f5-5dedef65518e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.275419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] MFA Rules not processed for user `9ea8a3d820c944248e7e4e5c4683e44e`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:19.276817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:19.280540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95799984-df92-40e0-84f5-5dedef65518e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.285564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.286417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.286702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.286977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.287727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.289363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.299854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 795/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.300836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.311593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d054-cad0-43ad-9169-79ec52e4206a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.316345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.316984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.317271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.317547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.318247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.319787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.331067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 796/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.331839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.340978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4432ca8-a569-4283-9d1c-b0e68aa008d7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.343995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 798/1594] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:19.344697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.345742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.346833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.346833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.347064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.347840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.349534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.354879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d2b04-aaea-4a14-97b2-14565be270e9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.359916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.360308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 797/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.360726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.361074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.361192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.361523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.362242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.364217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.371428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd8e5-2efe-4f7e-8927-e54e52a4ca74 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.375600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 799/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.375761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.377198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.377862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.378130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.378404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.379125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.380971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.388240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd8f37-367f-4664-9d9f-3a33f2ca99b6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.392645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 798/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.393348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.394949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:19.396751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.397693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.398062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.398368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] PATH_INFO: `/v3/projects/856d592872fa47a2b628f1c3e482b2fd/users/75f6d2e28f024875bda0b20f751bdf0d/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.412970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856d592872fa47a2b628f1c3e482b2fd, user_id=75f6d2e28f024875bda0b20f751bdf0d, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.415684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.452646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb15aa0-2f97-4589-974a-af927c319a40 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.459843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 800/1598] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/856d592872fa47a2b628f1c3e482b2fd/users/75f6d2e28f024875bda0b20f751bdf0d/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:19.460808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:19.464314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.504862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.505852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.506132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.506399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.526380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9cc40-2b04-4089-b82d-98244d627799 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.531016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.531676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.531965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.532241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.533018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.534618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.544560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 801/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.545434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.555370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320a15f-f00c-4859-8666-6f0e2ebcce35 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.559927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.560545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.560858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.561665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.562372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.563963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.569502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.570926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502de42-e7a0-4f1c-b04b-c97c1d407105 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.574142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 802/1600] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:19.574737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.583757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 799/1601] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:19.584744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcad94f-33d2-4810-bf6d-8400221456a4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.584954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecc08-d49f-4ec6-872e-5e3e0dc53d87 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.585333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecc08-d49f-4ec6-872e-5e3e0dc53d87 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.585522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecc08-d49f-4ec6-872e-5e3e0dc53d87 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.586913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 800/1602] 10.222.0.57 () {54 vars in 961 bytes} [Tue Jun 7 09:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:19.587461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecc08-d49f-4ec6-872e-5e3e0dc53d87 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.589898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.590761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.591039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.591436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] PATH_INFO: `/v3/projects/50189b9cf3b145ba84495af8db63c41b/users/2f32356bb0bd4234ba817db581b62aad/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.594648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:19.600098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecc08-d49f-4ec6-872e-5e3e0dc53d87 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.606471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.609682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.609682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.609682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] PATH_INFO: `/v3/projects/4e297b1a418941c0a90839a6d7ed8f89/users/b936366a1bc84badb0849df1c3fcc707/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.609942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50189b9cf3b145ba84495af8db63c41b, user_id=2f32356bb0bd4234ba817db581b62aad, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.613040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.622941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e297b1a418941c0a90839a6d7ed8f89, user_id=b936366a1bc84badb0849df1c3fcc707, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.626146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf80f3-e703-4731-aeee-51931dfebf7c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.736608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 803/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/50189b9cf3b145ba84495af8db63c41b/users/2f32356bb0bd4234ba817db581b62aad/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:19.736608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.743743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 801/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/4e297b1a418941c0a90839a6d7ed8f89/users/b936366a1bc84badb0849df1c3fcc707/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:19.745337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.745613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.745891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.764991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:19.766207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:19.808909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e41c2-9887-4dad-8d66-62e40ce3e202 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.830118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 802/1605] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:19.830937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.840546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f26a0a-f3d2-4ea3-b565-446207fc85ba None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:19.845638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.846521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.850797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.850797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] PATH_INFO: `/v3/projects/1a5f1ce85ba14353a6f13d1e6b4f11b2/users/057c5ce95a6b4aa48fc2eca3d3d963de/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.862418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5f1ce85ba14353a6f13d1e6b4f11b2, user_id=057c5ce95a6b4aa48fc2eca3d3d963de, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.865118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.869509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:19.870419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:19.870700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:19.870972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] PATH_INFO: `/v3/projects/61e2a8b0891548428759a52462a2feaa/users/06e30bbe21af428faa5ec0eab414abf5/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:19.885703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e2a8b0891548428759a52462a2feaa, user_id=06e30bbe21af428faa5ec0eab414abf5, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:19.888413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4e811-c3bc-47c4-bfb3-b9d6e675c092 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:19.973845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 803/1606] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/1a5f1ce85ba14353a6f13d1e6b4f11b2/users/057c5ce95a6b4aa48fc2eca3d3d963de/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:19.975942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:19.990906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:20.030675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 804/1607] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/61e2a8b0891548428759a52462a2feaa/users/06e30bbe21af428faa5ec0eab414abf5/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:20.032145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5537c5-45e5-426f-bb1e-40e738f058f9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.032375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5537c5-45e5-426f-bb1e-40e738f058f9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.032687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5537c5-45e5-426f-bb1e-40e738f058f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.041490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c581-2ad6-4d93-9ec8-585c59eaa955 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.054490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9f5537c5-45e5-426f-bb1e-40e738f058f9 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.054490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9f5537c5-45e5-426f-bb1e-40e738f058f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.103850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.105332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.105918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.106041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] PATH_INFO: `/v3/projects/1c21b1cbbd604d6d9c97ab7c3c788f3d/users/0346f4e3855142059983083e71c2c0c3/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.121350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c21b1cbbd604d6d9c97ab7c3c788f3d, user_id=0346f4e3855142059983083e71c2c0c3, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.124530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 805/1608] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.125073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.125958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82ebef-337a-4fda-83dd-cdc237e9c4fd None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.125958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82ebef-337a-4fda-83dd-cdc237e9c4fd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.126175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82ebef-337a-4fda-83dd-cdc237e9c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.133622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:20.145438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cd82ebef-337a-4fda-83dd-cdc237e9c4fd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.146650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cd82ebef-337a-4fda-83dd-cdc237e9c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.154531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 806/1609] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.155891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba17289-0f9f-44c8-af43-0fa3240828b4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.156193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba17289-0f9f-44c8-af43-0fa3240828b4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.156957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba17289-0f9f-44c8-af43-0fa3240828b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.185283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bba17289-0f9f-44c8-af43-0fa3240828b4 None None] MFA Rules not processed for user `b936366a1bc84badb0849df1c3fcc707`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.187015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bba17289-0f9f-44c8-af43-0fa3240828b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.209476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 804/1610] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:19 2022] PUT /identity/v3/projects/1c21b1cbbd604d6d9c97ab7c3c788f3d/users/0346f4e3855142059983083e71c2c0c3/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:20.211674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.266355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 807/1611] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.267645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1db3c-7ce9-42ad-be93-2f0bd7730358 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.267935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1db3c-7ce9-42ad-be93-2f0bd7730358 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.268211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1db3c-7ce9-42ad-be93-2f0bd7730358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.284818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baf2d5-6a88-49e4-8aec-2e25264a6e22 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.290568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.291271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.291984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.291984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.299171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2eb1db3c-7ce9-42ad-be93-2f0bd7730358 None None] MFA Rules not processed for user `057c5ce95a6b4aa48fc2eca3d3d963de`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.300491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2eb1db3c-7ce9-42ad-be93-2f0bd7730358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.366636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:20.369983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 808/1612] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.372088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.372486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.372917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.382823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.384740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.392508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.394148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.397406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 805/1613] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.398125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.408482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 809/1614] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.408482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.417877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750ce55-3d57-4384-bff7-986b437cf580 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.422972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.423268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.423512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.423933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.435813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.438225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8efe41-6c22-43ad-a146-914a9e279138 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.449084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 810/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.450578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976170b-11bf-4a36-bf5f-3c75059f8925 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.450936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976170b-11bf-4a36-bf5f-3c75059f8925 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.451278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976170b-11bf-4a36-bf5f-3c75059f8925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.470996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4976170b-11bf-4a36-bf5f-3c75059f8925 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.472457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4976170b-11bf-4a36-bf5f-3c75059f8925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.486482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 811/1616] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.487173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77174-6fd4-4d09-adc8-d2aa9a2ed931 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.488110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465be27-7919-4f74-8f26-50d2e1464e73 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.488402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465be27-7919-4f74-8f26-50d2e1464e73 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.488712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465be27-7919-4f74-8f26-50d2e1464e73 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.493818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.494702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.495146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 812/1617] 10.222.0.57 () {54 vars in 961 bytes} [Tue Jun 7 09:28:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:20.495280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.495594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.495815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465be27-7919-4f74-8f26-50d2e1464e73 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.557770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.562060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.562191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465be27-7919-4f74-8f26-50d2e1464e73 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.567333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.567994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.568200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.568471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.571437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 806/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.572341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.575478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:20.580765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.582394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55d02-7577-483e-8c5d-cb64f88d93f9 service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.582528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81127ea5-3d8c-4945-bb83-cbdbc4aa18eb service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.586487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.587225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.587562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.587845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.595464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 813/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.597048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.597271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.602298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.605114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.606725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.619839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.625472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.627121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 807/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.629740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.634938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 814/1621] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:20.648430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:20.654474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755438a6-f45c-4e51-874c-fd1ed5abeca7 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.658840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.659524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.660319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.660319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.672211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdcd48-a828-4e21-85cc-1a059ecedf0c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:20.686534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.687982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c24dd-4870-4820-926e-cc4cff08d933 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.688155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:20.688944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.689320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.689690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.719921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 808/1622] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.785803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:20.787690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125824bb-c2af-4e58-bde8-7a4d495f4507 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:20.812507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 815/1623] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:20.908016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da1c-946f-4c23-afbe-201470c9808c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.908016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da1c-946f-4c23-afbe-201470c9808c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.908306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da1c-946f-4c23-afbe-201470c9808c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.909997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 809/1624] 10.222.0.57 () {54 vars in 912 bytes} [Tue Jun 7 09:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:20.915626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5bb88-47c0-4e44-acaa-cc9e546bc3a7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:20.915906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5bb88-47c0-4e44-acaa-cc9e546bc3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:20.916274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5bb88-47c0-4e44-acaa-cc9e546bc3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:20.936859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8ff5bb88-47c0-4e44-acaa-cc9e546bc3a7 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:20.943744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8ff5bb88-47c0-4e44-acaa-cc9e546bc3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:20.948232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 816/1625] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:20.987366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98da1c-946f-4c23-afbe-201470c9808c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:21.012235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98da1c-946f-4c23-afbe-201470c9808c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:21.023515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:21.024184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.024488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.029169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.058432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:21.059963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd22ba-af41-4935-b99b-6a55376d24f8 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:21.082907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 810/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:21.148363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87720-c822-4968-ad00-dde0cbc22507 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.148948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87720-c822-4968-ad00-dde0cbc22507 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.148948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87720-c822-4968-ad00-dde0cbc22507 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.150352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 817/1627] 10.222.0.57 () {54 vars in 912 bytes} [Tue Jun 7 09:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:21.155292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.155808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.155905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.168877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a348f-51b3-4b13-8d36-a9583707cafd None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.169178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a348f-51b3-4b13-8d36-a9583707cafd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.169463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a348f-51b3-4b13-8d36-a9583707cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.205179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] MFA Rules not processed for user `9726e11bda974054a1e972c097b1ea1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:21.207242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:21.210800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-eb2a348f-51b3-4b13-8d36-a9583707cafd None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:21.220951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-eb2a348f-51b3-4b13-8d36-a9583707cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:21.253973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 818/1628] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:28:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:21.262037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:21.302777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 811/1629] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:21.304640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.304941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.305222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.337823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] MFA Rules not processed for user `9726e11bda974054a1e972c097b1ea1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:21.338984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:21.361700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 819/1630] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:21.775577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:21.789272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998224dc-5026-4a72-93e3-0dd21e9158b9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:21.794085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:21.795051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:21.795395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:21.796823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:21.810852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:21.812530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ba4f3-7265-46a0-9804-ffbe4813654f service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:21.824186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 812/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:22.482280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:22.499101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5af04-7e26-4623-9dfc-d89e42390161 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:22.506739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:22.507729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:22.508216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:22.508672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:22.518578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:22.520417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a323c-5d33-408d-ae37-99d514b12530 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:22.533212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 820/1632] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:23.258635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb90dc-2d0e-4827-b781-e3db940e630e None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:23.258635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb90dc-2d0e-4827-b781-e3db940e630e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:23.258635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb90dc-2d0e-4827-b781-e3db940e630e None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:23.258635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 813/1633] 10.222.0.57 () {54 vars in 912 bytes} [Tue Jun 7 09:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5212a6-7926-4489-8d7d-b2a88276649b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5212a6-7926-4489-8d7d-b2a88276649b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5212a6-7926-4489-8d7d-b2a88276649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6a5212a6-7926-4489-8d7d-b2a88276649b None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6a5212a6-7926-4489-8d7d-b2a88276649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:23.259278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 821/1634] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:23.259704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb90dc-2d0e-4827-b781-e3db940e630e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:23.259704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb90dc-2d0e-4827-b781-e3db940e630e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:23.259704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:23.259704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:23.260040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:23.260040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:23.260040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:23.260040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab08e1-6b50-4f0b-8f17-4b9f4d55d272 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:23.260040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 814/1635] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:24.292482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dce7b-d4ad-4c1b-80a7-f3fc3fca0287 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:24.293385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dce7b-d4ad-4c1b-80a7-f3fc3fca0287 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:24.293804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dce7b-d4ad-4c1b-80a7-f3fc3fca0287 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:24.296496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 822/1636] 10.222.0.57 () {54 vars in 960 bytes} [Tue Jun 7 09:28:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:24.302074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2ee-8bd8-42bc-ac01-1bc882527167 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:24.303031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2ee-8bd8-42bc-ac01-1bc882527167 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:24.303460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2ee-8bd8-42bc-ac01-1bc882527167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:24.325909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4ab0b2ee-8bd8-42bc-ac01-1bc882527167 None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:24.329030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4ab0b2ee-8bd8-42bc-ac01-1bc882527167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:24.440966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 815/1637] 10.222.0.57 () {58 vars in 1057 bytes} [Tue Jun 7 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:24.450643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dce7b-d4ad-4c1b-80a7-f3fc3fca0287 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:24.465958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dce7b-d4ad-4c1b-80a7-f3fc3fca0287 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:24.490672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:24.491344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:24.492823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:24.493111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:24.500190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:24.524314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:24.527315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:24.558446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 823/1638] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:30.560465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433070fc-e732-4aa2-9601-05d58244745c None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:30.561083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433070fc-e732-4aa2-9601-05d58244745c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:30.561483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433070fc-e732-4aa2-9601-05d58244745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:30.628350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-433070fc-e732-4aa2-9601-05d58244745c None None] MFA Rules not processed for user `0346f4e3855142059983083e71c2c0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:30.630200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-433070fc-e732-4aa2-9601-05d58244745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:30.838202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 816/1639] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:30.852673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:30.873667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f92b4-dd1a-41a7-8a13-b18626eeeb89 service placement] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:30.893155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:30.893934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:30.894281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:30.894633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:30.914488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:30.916394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e338d1-bb9f-4986-9f89-8442cbcf1438 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:30.940685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 824/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:34.250237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433070fc-e732-4aa2-9601-05d58244745c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:34.274289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433070fc-e732-4aa2-9601-05d58244745c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:34.284543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:34.285468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:34.314456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:34.314744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] PATH_INFO: `/v3/users/b936366a1bc84badb0849df1c3fcc707` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:34.325442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b936366a1bc84badb0849df1c3fcc707)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:34.341176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:34.616696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0cef-54f5-420f-907e-c1db0da869d0 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:34.617553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0cef-54f5-420f-907e-c1db0da869d0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:34.618155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0cef-54f5-420f-907e-c1db0da869d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:34.676814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f73d0cef-54f5-420f-907e-c1db0da869d0 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:34.678500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f73d0cef-54f5-420f-907e-c1db0da869d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:34.730764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:34.771148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b936366a1bc84badb0849df1c3fcc707'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:34.803973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b936366a1bc84badb0849df1c3fcc707'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:34.892179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 825/1641] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:34.896345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:34.899358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:34.899901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:34.938048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:34.956561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] MFA Rules not processed for user `2f32356bb0bd4234ba817db581b62aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:34.958050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:35.002845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b936366a1bc84badb0849df1c3fcc707'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:35.003683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b936366a1bc84badb0849df1c3fcc707'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:35.004748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17ac75c1-9998-4a5e-b657-c9a58cd66166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b936366a1bc84badb0849df1c3fcc707'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:35.046031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 817/1642] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:34 2022] DELETE /identity/v3/users/b936366a1bc84badb0849df1c3fcc707 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:35.048119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:35.048410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:35.049570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:35.130850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 826/1643] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:35.146315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:35.148024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:35.150511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:35.156937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:35.198168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7263-df1c-477a-b67c-638f720e8064 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:35.374801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 818/1644] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:35.376575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:35.378262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:35.454281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:35.454949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:35.455232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:35.455509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:35.573870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f17ef4-9dcf-40ad-9a98-d2f8cac71f7e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:35.588285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:35.589286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:35.589850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:35.593114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:35.617519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:35.625809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:35.651171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:35.665967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 819/1645] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:35.667025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:35.668536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:35.676339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:35.705218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 827/1646] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:35.902067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:35.967865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b699a-70fb-409c-a7e5-56ffaaab53f9 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:35.977334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:35.978303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:35.978725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:35.980120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:35.984433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:36.207384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:36.211637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a1d49-61e5-443c-841b-45911af4fc50 service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:36.239475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 820/1647] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:28:36.830780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:36.832557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:36.859521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6fd1-b74b-45a9-91d7-51b5bb2a4a55 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:36.876129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:36.877649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:36.878158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:36.881027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] PATH_INFO: `/v3/projects/4e297b1a418941c0a90839a6d7ed8f89` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:36.884054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e297b1a418941c0a90839a6d7ed8f89)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:36.887381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:37.147342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:37.199499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e297b1a418941c0a90839a6d7ed8f89', 'request_id': 'req-85cfa266-8208-42d3-8253-26fbe8d301eb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:37.231739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e297b1a418941c0a90839a6d7ed8f89', 'request_id': 'req-85cfa266-8208-42d3-8253-26fbe8d301eb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:37.268506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e297b1a418941c0a90839a6d7ed8f89', 'request_id': 'req-85cfa266-8208-42d3-8253-26fbe8d301eb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:37.411151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] The token cache is being invalidate because project 4e297b1a418941c0a90839a6d7ed8f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:28:37.412024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:37.414346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85cfa266-8208-42d3-8253-26fbe8d301eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:37.416813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 828/1648] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:28:36 2022] DELETE /identity/v3/projects/4e297b1a418941c0a90839a6d7ed8f89 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:37.421966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:37.422549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:37.422549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:37.482305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:37.482305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:37.691906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 821/1649] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:37.697470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:37.697470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:37.697470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:37.841537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:37.845717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:37.895786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:37.900135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:37.941885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 829/1650] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:37.943091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:37.944572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607bbf-9e27-4aae-8392-d39899efd45a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:37.962575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:37.963568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:37.963887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:37.964187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] PATH_INFO: `/v3/users/9ea8a3d820c944248e7e4e5c4683e44e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:37.968569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0062cd-ded6-4bbf-b00e-4dd7593e3fd4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:37.969883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea8a3d820c944248e7e4e5c4683e44e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:37.973863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:37.984399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:37.992064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:38.013043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 830/1651] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:38.017742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:38.064447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d168aae-8b47-4d1a-b5ee-0393c1e344bb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:38.091397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:38.092356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:38.092534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:38.092842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:38.093553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:38.095777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:38.245314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 831/1652] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:38 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:38.254928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:38.279165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea8a3d820c944248e7e4e5c4683e44e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:38.291394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea8a3d820c944248e7e4e5c4683e44e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:38.407788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea8a3d820c944248e7e4e5c4683e44e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:38.412901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716807-2079-4efa-a555-0d0e6aa384bf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:38.413162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea8a3d820c944248e7e4e5c4683e44e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:38.413162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-afa42ed5-6a0d-4080-b1db-161909101724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea8a3d820c944248e7e4e5c4683e44e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:38.414817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:38.415667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:38.416134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:38.416548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:38.417409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:38.423222 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:38.463638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 822/1653] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:37 2022] DELETE /identity/v3/users/9ea8a3d820c944248e7e4e5c4683e44e => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:28:38.468363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc9f6-f632-46c8-a6e9-1c93b9ea5abb None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:38.468846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc9f6-f632-46c8-a6e9-1c93b9ea5abb None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:38.469489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc9f6-f632-46c8-a6e9-1c93b9ea5abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:38.523781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4dfcc9f6-f632-46c8-a6e9-1c93b9ea5abb None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:38.528601 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4dfcc9f6-f632-46c8-a6e9-1c93b9ea5abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:38.540976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 832/1654] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:38 2022] POST /identity/v3/users => generated 284 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:38.545550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:38.547577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:38.555075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:38.598295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a83b1-48a8-4492-a7a9-deb442b6d9f8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:38.713692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 823/1655] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:38.715181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b907c-5bf6-40ce-9832-77d3c39d7e80 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:38.715516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b907c-5bf6-40ce-9832-77d3c39d7e80 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:38.715800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b907c-5bf6-40ce-9832-77d3c39d7e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:38.784526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:38.785656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:38.785935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:38.786371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:38.787372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:38.789224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:38.802737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 833/1656] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:28:38.809752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:38.809992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e32b907c-5bf6-40ce-9832-77d3c39d7e80 None None] MFA Rules not processed for user `06e30bbe21af428faa5ec0eab414abf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:38.809992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e32b907c-5bf6-40ce-9832-77d3c39d7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:38.810145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:38.819782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:39.052725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be8688-44a5-4877-8c7b-6f0c745c5411 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:39.061548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:39.062220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.062507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.062788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.084741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:39.094377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:39.105243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 824/1657] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:39.106931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.108219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.108219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.124117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 834/1658] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:39.125023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:39.152413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971a7e-ddb1-4dfc-826c-dffde52e8039 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:39.165482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:39.166228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.166519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.166801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.167608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:39.169280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:39.191892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 835/1659] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:39.191892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:39.198253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:39.221384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] MFA Rules not processed for user `75f6d2e28f024875bda0b20f751bdf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:39.222631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:39.397301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 825/1660] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:39.398688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:39.429854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebe99b-5cd9-41f1-abce-142239df722b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:39.432434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4874ef-75cd-43cb-aa9e-c617e0b4b9f3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:39.439830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:39.441049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.441818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.442331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] PATH_INFO: `/v3/projects/39ca520b83124fc194954dce333920db/users/a3078cea5a3144c99063e532a6c7d2c5/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.453357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:39.461405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.461405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.461405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.468910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ca520b83124fc194954dce333920db, user_id=a3078cea5a3144c99063e532a6c7d2c5, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:39.474301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:39.475635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:39.479943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76007f-9884-4ba4-9b28-03a08c1d1655 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:39.502607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 836/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:39.504559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baeb20-684a-4b5b-9cd4-8da328aaf87a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.504880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baeb20-684a-4b5b-9cd4-8da328aaf87a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.505174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baeb20-684a-4b5b-9cd4-8da328aaf87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.530580 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d9baeb20-684a-4b5b-9cd4-8da328aaf87a None None] MFA Rules not processed for user `75f6d2e28f024875bda0b20f751bdf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:39.532074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d9baeb20-684a-4b5b-9cd4-8da328aaf87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:39.540151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 837/1662] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:39.551985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 826/1663] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:39 2022] PUT /identity/v3/projects/39ca520b83124fc194954dce333920db/users/a3078cea5a3144c99063e532a6c7d2c5/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:39.567285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a54c-6c25-4315-9556-90f7ec65eab9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.567608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a54c-6c25-4315-9556-90f7ec65eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.567883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a54c-6c25-4315-9556-90f7ec65eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.659298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-62c4a54c-6c25-4315-9556-90f7ec65eab9 None None] MFA Rules not processed for user `a3078cea5a3144c99063e532a6c7d2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:39.664254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-62c4a54c-6c25-4315-9556-90f7ec65eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:39.748426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:39.831503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 838/1664] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:39.833455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:39.833754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:39.834029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:39.862205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:39.863798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:39.994065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 839/1665] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Jun 7 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:39.997892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:40.008569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803560-f582-4a81-af52-b648662b5a04 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.024795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.027445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.028669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.030107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.135318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:40.214477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673eb13-b810-4910-b988-b3e9e532e8c9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.228999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:40.230405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.231263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.231545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.233458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.235784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:40.236515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:40.238481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:40.261713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 827/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:40.267238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:40.274126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:40.307508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9a639-828c-42a9-824a-bb37d1f1a48b service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.316785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.317348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.317653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.317895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.344815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:40.347263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:40.375537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 828/1667] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:40.412447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 840/1668] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:39 2022] POST /identity/v3/projects => generated 371 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:40.416778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:40.432311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e410-99ab-4bfb-b828-5f0f16cc5e60 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.522596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:40.589818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.593833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.594109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.594394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.595108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:40.597470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:40.823882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 829/1669] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:40 2022] POST /identity/v3/users => generated 292 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:40.827855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:40.844971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975883d-621c-406b-8a20-8fb5269dbd69 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.861758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.863342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.863818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.864888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.867836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:40.871377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e49b3b-2e3c-4586-b017-041cbc3d85b7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:40.919280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 841/1670] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:40.921831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:40.951720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b4f20-e167-4486-899b-5eba36443bbb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:40.964471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:40.966797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.969103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.969364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] PATH_INFO: `/v3/projects/70fa4278518e4cafba3e218f5ccff441/users/5e84125f691f4da48a5c45f308275c90/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:40.974022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:40.974191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:40.977489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:41.043779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] MFA Rules not processed for user `0346f4e3855142059983083e71c2c0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:41.045598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:41.053650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fa4278518e4cafba3e218f5ccff441, user_id=5e84125f691f4da48a5c45f308275c90, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:41.071803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:41.191606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 830/1671] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:40 2022] PUT /identity/v3/projects/70fa4278518e4cafba3e218f5ccff441/users/5e84125f691f4da48a5c45f308275c90/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:41.195412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:41.325455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 842/1672] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:41.325455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:41.334294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:41.402282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5b6c2-8a8d-416d-85df-a2db08e137dd None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:41.428718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:41.429642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:41.434499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:41.434499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] PATH_INFO: `/v3/projects/02c52b4e82354dd5af2f6daa818b6753` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:41.434754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23c125-9690-4715-8edd-71d19224bd72 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:41.436348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c52b4e82354dd5af2f6daa818b6753)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:41.442598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:41.448440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:41.449155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:41.449494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:41.449730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:41.450473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:41.462618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:41.496956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 831/1673] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:41.497958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:41.529127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4db31-5f28-450b-be0b-3c08c67a80b3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:41.562313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:41.575662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:41.575859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:41.576699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] PATH_INFO: `/v3/projects/70fa4278518e4cafba3e218f5ccff441/users/5e84125f691f4da48a5c45f308275c90/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:41.590421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fa4278518e4cafba3e218f5ccff441, user_id=5e84125f691f4da48a5c45f308275c90, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:41.594801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae7ccb-93e1-4385-92a2-f8b0be9bd667 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:41.618849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 832/1674] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:41 2022] PUT /identity/v3/projects/70fa4278518e4cafba3e218f5ccff441/users/5e84125f691f4da48a5c45f308275c90/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:41.620525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:41.620847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:41.621123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:41.679287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] MFA Rules not processed for user `5e84125f691f4da48a5c45f308275c90`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:41.679287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:41.711589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:41.740848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:41.796180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c52b4e82354dd5af2f6daa818b6753', 'request_id': 'req-149f248d-276f-42c3-ba12-5652521f5a3e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:41.797753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 833/1675] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:41.806562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:41.809586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:41.860903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c52b4e82354dd5af2f6daa818b6753', 'request_id': 'req-149f248d-276f-42c3-ba12-5652521f5a3e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:41.894032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c52b4e82354dd5af2f6daa818b6753', 'request_id': 'req-149f248d-276f-42c3-ba12-5652521f5a3e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:42.020011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbccfae-3543-43e6-b50b-5251ac85d78f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:42.068518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] The token cache is being invalidate because project 02c52b4e82354dd5af2f6daa818b6753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:28:42.068518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:42.068518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-149f248d-276f-42c3-ba12-5652521f5a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:42.068518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 843/1676] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:28:41 2022] DELETE /identity/v3/projects/02c52b4e82354dd5af2f6daa818b6753 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:42.075441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.075747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.076039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.142548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:42.147506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.147957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.148493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.150751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:42.161931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:42.163523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:42.430981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:42.432744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd2622-d5c9-4b03-a759-2a7e71db8159 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:42.449421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 844/1677] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:42.455789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:42.462722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:42.484809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 834/1678] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:42.486843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce37e-c633-4a47-8797-cd85c5fb3771 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.488951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce37e-c633-4a47-8797-cd85c5fb3771 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.493931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce37e-c633-4a47-8797-cd85c5fb3771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.535786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d5a0-c111-4ae2-9678-bcec36f9b983 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:42.549866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:42.556145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.558790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.559120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] PATH_INFO: `/v3/users/057c5ce95a6b4aa48fc2eca3d3d963de` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.559306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-190ce37e-c633-4a47-8797-cd85c5fb3771 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:42.564453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057c5ce95a6b4aa48fc2eca3d3d963de)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:42.565397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-190ce37e-c633-4a47-8797-cd85c5fb3771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:42.571670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: WARNING py.warnings [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:42.571670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: warnings.warn(msg) Jun 07 09:28:42.571670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: Jun 07 09:28:42.578661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:42.589996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 835/1679] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:42.591545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.594906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.594906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.649430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] MFA Rules not processed for user `2f32356bb0bd4234ba817db581b62aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:42.653845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:42.904567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 836/1680] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:42.909778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:42.953546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf967ed8-4f51-42b9-85c2-ca2d9516d30f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:42.964459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:42.978915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:42.983001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:42.984191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:42.987224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:42.990498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:42.991562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:42.997960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:43.001620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057c5ce95a6b4aa48fc2eca3d3d963de'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:43.029782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057c5ce95a6b4aa48fc2eca3d3d963de'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:43.032981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057c5ce95a6b4aa48fc2eca3d3d963de'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:43.037609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 837/1681] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:43.041515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:43.091756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e2b8-ee1a-4ad4-bb43-2631b125fb27 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:43.104068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:43.104795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:43.105080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:43.105411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:43.106283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:43.109199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:43.185190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057c5ce95a6b4aa48fc2eca3d3d963de'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:43.186480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c47c0d7-e312-418f-84d7-b31bea99dab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057c5ce95a6b4aa48fc2eca3d3d963de'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:43.266037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 845/1682] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:42 2022] DELETE /identity/v3/users/057c5ce95a6b4aa48fc2eca3d3d963de => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:43.267540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:43.267830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:43.268237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:43.281029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 838/1683] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:43 2022] POST /identity/v3/projects => generated 355 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:28:43.293588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:43.300635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:43.354056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:43.362326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:43.585080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a8ac-28a4-45b3-9a5d-14293c0f0b02 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:43.604305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:43.608341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:43.611047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:43.611385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:43.613766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:43.614582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:43.627226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 846/1684] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:43.629544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:43.762748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 839/1685] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:43 2022] POST /identity/v3/users => generated 278 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:43.764975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:43.780617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd6d6c-6801-445e-bb1a-e009f96ffb1b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:43.787318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:43.788380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:43.788701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:43.789114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:43.790774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:43.793001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:43.833670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:43.838282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:43.839785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 847/1686] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:43.856530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:43.907803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad089-cded-4b07-8de9-dd4f0f530813 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:43.935799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:43.940365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:43.942894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:43.944751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:43.949674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:43.954401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:44.031918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 848/1687] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:44.031918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:44.073679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769663d8-ad42-4f66-b6f5-d5810ce13e95 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:44.105389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:44.106299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.106738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.106959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] PATH_INFO: `/v3/projects/8b8587f3ac1e42a89a81d91c7a515d31/users/7523b43aa7f54b3f9e92139486fad6da/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.133886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bc721-8eca-45bb-b7c1-39b447f72ff9 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:44.164961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:44.168219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.168219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.173682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.178541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8587f3ac1e42a89a81d91c7a515d31, user_id=7523b43aa7f54b3f9e92139486fad6da, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:44.183544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea97631-87be-44a9-9894-05fe1e49c49a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:44.201914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:44.209705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f2817-ba0f-47f0-9f55-d5abe358984a service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:44.254456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 840/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:44.308191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 849/1689] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:44 2022] PUT /identity/v3/projects/8b8587f3ac1e42a89a81d91c7a515d31/users/7523b43aa7f54b3f9e92139486fad6da/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:44.309917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094214a6-069b-4e06-89cf-610e22ed64be None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.310061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094214a6-069b-4e06-89cf-610e22ed64be None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.310284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094214a6-069b-4e06-89cf-610e22ed64be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.367735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.367735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.367735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.370438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 850/1690] 10.222.0.57 () {54 vars in 912 bytes} [Tue Jun 7 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:28:44.370438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:44.371953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:44.381744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-094214a6-069b-4e06-89cf-610e22ed64be None None] MFA Rules not processed for user `7523b43aa7f54b3f9e92139486fad6da`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:44.383053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-094214a6-069b-4e06-89cf-610e22ed64be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:44.475531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:44.533565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7523af1-d77d-4093-8f1f-3f4837d620c2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:44.546616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:44.547502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.547785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.548058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] PATH_INFO: `/v3/users/a3078cea5a3144c99063e532a6c7d2c5` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.556153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3078cea5a3144c99063e532a6c7d2c5)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:44.557857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:44.639334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 841/1691] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:44.658247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.658967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.659177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.725594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:44.725594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:44.874450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 842/1692] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:44.875266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:44.898442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7577b-190c-438a-b889-563866a2ae07 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:44.908742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:44.909501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:44.909779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:44.910058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:44.910768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:44.912761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:45.029510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3078cea5a3144c99063e532a6c7d2c5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:45.040336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3078cea5a3144c99063e532a6c7d2c5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:45.041364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3078cea5a3144c99063e532a6c7d2c5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:45.055792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 843/1693] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:44 2022] POST /identity/v3/projects => generated 357 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:45.060169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:45.063236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.169469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3078cea5a3144c99063e532a6c7d2c5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:45.170602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3078cea5a3144c99063e532a6c7d2c5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:45.222409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 851/1694] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:44 2022] DELETE /identity/v3/users/a3078cea5a3144c99063e532a6c7d2c5 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:45.226395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:45.235379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.244266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ff0c6-c7bc-4007-b0b4-cdd923a0782f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:45.444574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:45.446772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.452738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.452738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.466733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.495504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf983d-34c8-4661-8e37-22e7c65e6d8f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:45.526619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:45.531343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bdfd6-e4dc-4c2c-aa34-7b3698ffed84 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:45.538882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:45.542102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.542102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.542102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.542102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:45.551295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67eb99-eaf4-4b09-8457-d8ccc8a53929 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:45.564524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 844/1695] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:45.565869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3c27-d18f-403f-a511-1acf45df2ba7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.566148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3c27-d18f-403f-a511-1acf45df2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.566531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3c27-d18f-403f-a511-1acf45df2ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.628708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1b5d3c27-d18f-403f-a511-1acf45df2ba7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:45.629997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1b5d3c27-d18f-403f-a511-1acf45df2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.654728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 845/1696] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:45.656411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.656411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.656724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.660885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 846/1697] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:28:45.678915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:45.686228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.721544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 852/1698] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:45 2022] POST /identity/v3/users => generated 285 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:45.725723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.725723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.725723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.726008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f96ee3-9b28-4ec8-a7c9-d9ae17f31a73 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:45.737212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:45.750392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.753168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.753460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.796755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:45.798267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:45.803794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:45.805095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:45.831982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 847/1699] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:45.835585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:45.851934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadd968-8426-41f0-a112-ec9100abf32d service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:45.872112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:45.873064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:45.873453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:45.874139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:45.877462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:45.879225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:45.955374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 848/1700] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:45.960308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:45.973086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 853/1701] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:45.981648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd493-1abc-4da5-ae0f-e0d9d90a1b51 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:46.028716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:46.028716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:46.028716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:46.028716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] PATH_INFO: `/v3/projects/23b032f9567a4dfa9d34599bb320d765/users/54167db9f3484d5488a029bee34e4a84/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:46.083126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b032f9567a4dfa9d34599bb320d765, user_id=54167db9f3484d5488a029bee34e4a84, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:46.088686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:46.437976 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:46.488732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 849/1702] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:45 2022] PUT /identity/v3/projects/23b032f9567a4dfa9d34599bb320d765/users/54167db9f3484d5488a029bee34e4a84/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:28:46.493533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:46.508191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253c1d-06ff-4393-b8e1-941b3d933e28 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:46.581043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:46.690277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:46.691223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:46.692173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:46.693464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:46.694451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:46.698844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc6756-5c08-424c-9bf3-17d433f52106 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:46.727109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:46.727660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 854/1703] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:46.734112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b33d94-b11c-4613-8262-3bc1458113b1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:46.740896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:46.742040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:46.742497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:46.742980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] PATH_INFO: `/v3/projects/23b032f9567a4dfa9d34599bb320d765/users/54167db9f3484d5488a029bee34e4a84/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:46.745949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b032f9567a4dfa9d34599bb320d765, user_id=54167db9f3484d5488a029bee34e4a84, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:46.750225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bfb76-9688-489d-bb30-b975a7bb80e5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:46.773468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 850/1704] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:46 2022] PUT /identity/v3/projects/23b032f9567a4dfa9d34599bb320d765/users/54167db9f3484d5488a029bee34e4a84/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:28:46.787323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:46.787615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:46.788630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:46.845006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c907d3-83cc-4d74-ac2f-ab1165f23d18 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:46.845301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c907d3-83cc-4d74-ac2f-ab1165f23d18 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:46.845584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c907d3-83cc-4d74-ac2f-ab1165f23d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:46.876886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] MFA Rules not processed for user `54167db9f3484d5488a029bee34e4a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:46.878496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:46.895139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d3c907d3-83cc-4d74-ac2f-ab1165f23d18 None None] MFA Rules not processed for user `06e30bbe21af428faa5ec0eab414abf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:46.896388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d3c907d3-83cc-4d74-ac2f-ab1165f23d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:47.116246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 851/1705] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:47.117874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:47.119613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:47.119613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:47.149449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 855/1706] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:47.166333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:47.168094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:47.201840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] MFA Rules not processed for user `75f6d2e28f024875bda0b20f751bdf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:47.211859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:47.389760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9748e-9466-40b6-8ac9-64d6d3ab544b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:47.409570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:47.413555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:47.416206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:47.417468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:47.444663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 852/1707] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:47.461392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:47.462969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:47.513016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 856/1708] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:47.522889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:47.526258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:47.662138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:47.664054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:47.747228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aabec-8fce-43fd-ad20-867bf9ada73c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:47.767972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:47.769191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:47.769718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:47.770307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] PATH_INFO: `/v3/projects/1a5f1ce85ba14353a6f13d1e6b4f11b2` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:47.772408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5f1ce85ba14353a6f13d1e6b4f11b2)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:47.773746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: WARNING py.warnings [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 09:28:47.773746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: warnings.warn(msg) Jun 07 09:28:47.773746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: Jun 07 09:28:47.775530 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:47.936273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55891cba-f75c-46f0-83d4-7cf6c6af0f23 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:47.976172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5f1ce85ba14353a6f13d1e6b4f11b2', 'request_id': 'req-be1933e9-326d-4d14-8828-4d657769c229'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.016509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5f1ce85ba14353a6f13d1e6b4f11b2', 'request_id': 'req-be1933e9-326d-4d14-8828-4d657769c229'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.028086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5f1ce85ba14353a6f13d1e6b4f11b2', 'request_id': 'req-be1933e9-326d-4d14-8828-4d657769c229'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.147035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:48.148206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.148206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.148373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.152107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] The token cache is being invalidate because project 1a5f1ce85ba14353a6f13d1e6b4f11b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:28:48.152611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.153718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.159597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 853/1709] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:28:47 2022] DELETE /identity/v3/projects/1a5f1ce85ba14353a6f13d1e6b4f11b2 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:48.159597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:48.169157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.189701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:48.191640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:48.216936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 857/1710] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:48.218398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:48.234334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.379876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:48.434057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1933e9-326d-4d14-8828-4d657769c229 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:48.448597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:48.449670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.450184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.450665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.454373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.522176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749d5f6-edad-4003-a7df-5a9dbc6306d7 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:48.540634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:48.541775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.542066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.542349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] PATH_INFO: `/v3/projects/39ca520b83124fc194954dce333920db` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.546608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:48.547520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ca520b83124fc194954dce333920db)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:48.548484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bfd32-8f9a-48d5-8f6e-87a5c5157ba9 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:48.549661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:48.570374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 854/1711] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:48 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:48.571951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354edba-6ca9-4146-bf20-75ced6ec3ab2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.572395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354edba-6ca9-4146-bf20-75ced6ec3ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.572850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354edba-6ca9-4146-bf20-75ced6ec3ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.598366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5354edba-6ca9-4146-bf20-75ced6ec3ab2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:48.599949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5354edba-6ca9-4146-bf20-75ced6ec3ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.611652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 855/1712] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:48.626220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218575c-062a-4221-a8ce-68696b03baed None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.626513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218575c-062a-4221-a8ce-68696b03baed None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.626796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218575c-062a-4221-a8ce-68696b03baed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.663824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f218575c-062a-4221-a8ce-68696b03baed None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:48.673214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f218575c-062a-4221-a8ce-68696b03baed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.677654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 856/1713] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:48.683748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218575c-062a-4221-a8ce-68696b03baed None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:48.731817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ca520b83124fc194954dce333920db', 'request_id': 'req-2e13b2dc-8771-4a8e-a267-f75e8d935de5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.741662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ca520b83124fc194954dce333920db', 'request_id': 'req-2e13b2dc-8771-4a8e-a267-f75e8d935de5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.758560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ca520b83124fc194954dce333920db', 'request_id': 'req-2e13b2dc-8771-4a8e-a267-f75e8d935de5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.762199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218575c-062a-4221-a8ce-68696b03baed None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:48.876175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:48.883126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] The token cache is being invalidate because project 39ca520b83124fc194954dce333920db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:28:48.890343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.890343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:48.890343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 858/1714] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:28:48 2022] DELETE /identity/v3/projects/39ca520b83124fc194954dce333920db => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:48.891837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:48.894625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:48.981458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:48.988182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:48.988738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:48.989125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:48.990316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:48.990705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:48.999457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.041999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 857/1715] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:28:49.047049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:49.049386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:49.050286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b2dc-8771-4a8e-a267-f75e8d935de5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.056988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.058802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.059368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.060053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] PATH_INFO: `/v3/users/5e84125f691f4da48a5c45f308275c90` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.061890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e84125f691f4da48a5c45f308275c90)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.064403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.082001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3b5cb-3143-4116-87f9-823a201cc39e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.095007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.096775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.097514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.100187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.115476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.120531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.285079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 858/1716] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:49 2022] POST /identity/v3/projects => generated 363 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:49.287952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:49.392385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e84125f691f4da48a5c45f308275c90'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:49.413715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e84125f691f4da48a5c45f308275c90'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:49.417206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e84125f691f4da48a5c45f308275c90'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:49.532129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e84125f691f4da48a5c45f308275c90'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:49.533229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e84125f691f4da48a5c45f308275c90'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:49.559396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec65-9516-4d46-bc45-65593bcf3c6e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.566081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.567059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.567480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.567909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.568810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.574778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.592502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 859/1717] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:48 2022] DELETE /identity/v3/users/5e84125f691f4da48a5c45f308275c90 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:49.687329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 859/1718] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:49 2022] POST /identity/v3/users => generated 282 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:49.705190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:49.705190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:49.760474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:49.784926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ada7a-43d3-4b84-88e1-552f6ff01900 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.806093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.807144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.807616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.808242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.810581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.812706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.825766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 860/1719] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:28:49.830342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:49.847376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b5484-f368-4dcd-a6c0-66b98795fde8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.854964 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.856121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.856768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.858228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.859792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.863720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb2a4-80d8-4f19-994c-cb992767982e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:49.880251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 860/1720] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:28:49.885647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:49.913437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589293-8cf9-41a7-98fd-085f343779d3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:49.920926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:49.922377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:49.923093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:49.924263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] PATH_INFO: `/v3/projects/2ff66bebe7064c9fa058deb52fdfcc31/users/42315cae58e44bfdb6e7401dfbcd8688/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:49.946125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff66bebe7064c9fa058deb52fdfcc31, user_id=42315cae58e44bfdb6e7401dfbcd8688, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:49.954606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:50.303871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 861/1721] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:49 2022] PUT /identity/v3/projects/2ff66bebe7064c9fa058deb52fdfcc31/users/42315cae58e44bfdb6e7401dfbcd8688/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:50.311421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:50.311712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:50.311993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:50.382007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] MFA Rules not processed for user `42315cae58e44bfdb6e7401dfbcd8688`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:50.384421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:50.581562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 861/1722] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:50.583290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:50.812544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b3cd-2e19-4b12-8ae8-2589eb834e5d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:50.838591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:50.839380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:50.839738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:50.840106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:50.841007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:50.860241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:51.418931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 862/1723] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:50 2022] POST /identity/v3/projects => generated 365 bytes in 811 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:51.420164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:51.439571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d77af-a7cd-4c17-817c-e458d0b7df5a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:51.613818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:51.877176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:51.877176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:51.877176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:51.877176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:51.877795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:51.893876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:52.081506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 862/1724] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:51 2022] POST /identity/v3/users => generated 289 bytes in 667 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:52.085372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:52.111701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e196e-cb2f-42d1-af79-22fd604da435 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:52.120480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:52.121214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:52.121500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:52.121779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:52.122522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:52.125388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:52.205146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 863/1725] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:52.210548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:52.226731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60029b-89c6-4c3b-a825-5754b947d48a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:52.236936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:52.238002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:52.238584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:52.238893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] PATH_INFO: `/v3/projects/0cd554a672414f6d99872135eba8768c/users/6908775194b741efbe14636c714cedca/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:52.258932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd554a672414f6d99872135eba8768c, user_id=6908775194b741efbe14636c714cedca, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:52.263378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:52.377957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:52.380449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:52.422847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 863/1726] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:52 2022] PUT /identity/v3/projects/0cd554a672414f6d99872135eba8768c/users/6908775194b741efbe14636c714cedca/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:52.427206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:52.462804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f992ef2-ac25-41fd-91d4-8bc7308f2fae admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:52.651045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:52.653231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:52.653669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:52.654161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] PATH_INFO: `/v3/projects/70fa4278518e4cafba3e218f5ccff441` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:52.655841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fa4278518e4cafba3e218f5ccff441)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:52.658750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:52.763931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c33c5-aa2a-4c20-9919-715d23e3ef7d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:52.774962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:52.776840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:52.778765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:52.779386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:52.788151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:52.792296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:52.814659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 864/1727] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:52.820412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:52.834038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee568c-20ca-4a81-a691-2dbfb06bbfe6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:52.872263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fa4278518e4cafba3e218f5ccff441', 'request_id': 'req-4323ebd9-b45d-43f1-841f-b964a44c5423'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:52.895730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fa4278518e4cafba3e218f5ccff441', 'request_id': 'req-4323ebd9-b45d-43f1-841f-b964a44c5423'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:52.911518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fa4278518e4cafba3e218f5ccff441', 'request_id': 'req-4323ebd9-b45d-43f1-841f-b964a44c5423'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:53.009979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:53.038943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] The token cache is being invalidate because project 70fa4278518e4cafba3e218f5ccff441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:28:53.039244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:53.044186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4323ebd9-b45d-43f1-841f-b964a44c5423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:28:53.058864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 864/1728] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:28:52 2022] DELETE /identity/v3/projects/70fa4278518e4cafba3e218f5ccff441 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:28:53.064335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f96af-95d1-452c-8084-edd02c958c18 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.064675 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f96af-95d1-452c-8084-edd02c958c18 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.064940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f96af-95d1-452c-8084-edd02c958c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.070150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.072981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.072981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.072981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] PATH_INFO: `/v3/projects/0cd554a672414f6d99872135eba8768c/users/6908775194b741efbe14636c714cedca/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.077088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd554a672414f6d99872135eba8768c, user_id=6908775194b741efbe14636c714cedca, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.081984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7313a-f8b1-48e7-a699-9e880092a341 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:53.111583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a78f96af-95d1-452c-8084-edd02c958c18 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:53.113252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a78f96af-95d1-452c-8084-edd02c958c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:53.132218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 865/1729] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:52 2022] PUT /identity/v3/projects/0cd554a672414f6d99872135eba8768c/users/6908775194b741efbe14636c714cedca/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:53.145018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.145637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.145899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.181933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] MFA Rules not processed for user `6908775194b741efbe14636c714cedca`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:53.183819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:53.255226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 865/1730] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:53.266024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.266520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.268434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.327733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 866/1731] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:28:53.329762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:53.349083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:53.349697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.349697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:53.354223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 866/1732] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:53.359127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.379923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d888-f29d-4ba9-b63a-df5f02fcb859 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.394236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.394920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.395296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.395636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.396379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:28:53.396716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.398339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:53.428694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 867/1733] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:28:53.433039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.453227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc6c49-c76f-4e08-85e4-8151a1db3bf6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.459980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.460707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.461001 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.461280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.461997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.464110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:53.516148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fe032-413b-466b-8a37-4b86a1d2694d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.521520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.521520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.521520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.522223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.532902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.534910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:53.550687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 867/1734] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:53.639343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 868/1735] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:53 2022] POST /identity/v3/projects => generated 369 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:53.641867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.657727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8349150-ed29-4e53-a0d1-2024281878e6 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.819663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.820347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.820651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.826218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.826218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.826218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:53.837998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.907510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c660-92e3-470a-a7c8-bf8560208393 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.911511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.912293 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.912715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.920120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.920120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:53.945895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 868/1736] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:53 2022] POST /identity/v3/users => generated 285 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:53.949571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:53.963005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec581-2031-42eb-af4b-dd6a72b4ff84 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:53.980940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:53.981512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:53.981804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:53.982752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:53.983952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:53.987430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.007861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 869/1737] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:28:54.017886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.022027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.023945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.028276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08a89f-e673-4543-a4a7-6ad970249b73 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.035895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.036609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 869/1738] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:54.037104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.037490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.037770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.038743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.043789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626cbdd-5f81-4c4c-8a55-e4080507681b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.061695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 870/1739] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:54.065819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.080497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05605c-4d03-4ca8-867e-b2d9ac82432d service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.086954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.088248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.088699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.089049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] PATH_INFO: `/v3/projects/6f93ad57f88f410296f38953d870d002/users/5b2ce2786157469f818ff9c8216433ef/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.095574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025b6f1-667a-4c24-95ee-39674524b447 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.101380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025b6f1-667a-4c24-95ee-39674524b447 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.101380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025b6f1-667a-4c24-95ee-39674524b447 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.101815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 871/1740] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:28:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:28:54.121188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f93ad57f88f410296f38953d870d002, user_id=5b2ce2786157469f818ff9c8216433ef, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.124492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f72a3-e43e-417d-ac66-b2fc16f660a8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.134574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.134867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.135146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.163754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:54.165029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:54.252669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 870/1741] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:54 2022] PUT /identity/v3/projects/6f93ad57f88f410296f38953d870d002/users/5b2ce2786157469f818ff9c8216433ef/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:54.268218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.268332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.268569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.337742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] MFA Rules not processed for user `5b2ce2786157469f818ff9c8216433ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:54.338730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 872/1742] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:28:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:54.339712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:54.370962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.373377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:54.392958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528b576-ec09-4803-b9b8-d4dcd6c9896e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.443709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 871/1743] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:54.448261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.449154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:54.477688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.478604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.478780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.479065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.489157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.490987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.505098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 873/1744] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:28:54.549897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8b7fa-68b5-4f9b-9517-fe495fa72eb4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.555415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.556220 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.556355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.556662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.557359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.559250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.689442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 872/1745] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:28:54 2022] POST /identity/v3/projects => generated 367 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:54.692445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.703750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66b99c-099e-4932-afff-654c8e8beb98 service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.762518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.769497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.769497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.769497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.769497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.769497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.853346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 874/1746] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:28:54 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:28:54.858170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.874484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06de-b458-49db-b406-abad83ef7656 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.879556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.880300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.880690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.881051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.881882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.883607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:54.895304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 873/1747] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:54.901119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:54.910345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513259a-19af-4a05-9528-317ebcabb806 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:54.914914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:54.915783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:54.916062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:54.916333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] PATH_INFO: `/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388/users/f0988ee57e0b411ea7f1a023edfe508f/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:54.931590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7ff5e8188e483c8b4ac61db2c3b388, user_id=f0988ee57e0b411ea7f1a023edfe508f, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:54.934305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:55.036807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 875/1748] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:54 2022] PUT /identity/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388/users/f0988ee57e0b411ea7f1a023edfe508f/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:28:55.041921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:55.052707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcba52-ed81-4632-91b6-f07f29982983 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:55.112983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:55.113735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:55.114394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:55.125083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:55.126195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:55.128074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eacec-8cfb-43b6-9902-90ddad290ec9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:55.140357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 874/1749] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:28:55.144528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:55.158935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf58f67-3750-4449-b6da-43b6031c2ba4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:55.164078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:55.165020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:55.165311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:55.165869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] PATH_INFO: `/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388/users/f0988ee57e0b411ea7f1a023edfe508f/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:55.168272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7ff5e8188e483c8b4ac61db2c3b388, user_id=f0988ee57e0b411ea7f1a023edfe508f, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:55.171139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:55.198961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 876/1750] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:28:55 2022] PUT /identity/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388/users/f0988ee57e0b411ea7f1a023edfe508f/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:28:55.200313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:55.201274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:55.201691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:55.234142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] MFA Rules not processed for user `f0988ee57e0b411ea7f1a023edfe508f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:28:55.235404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:55.316328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 875/1751] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:28:55.332281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:55.334101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:28:55.411644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66b3f-7c52-490e-a4d8-9a72718d5c0f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:55.417777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:55.419139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:55.419766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:55.420269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:55.433549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:55.438134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066d95e-402c-4f25-bc49-be5b17042d22 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:28:55.472373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 877/1752] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:28:56.417286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:56.497513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:58.047818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:58.979825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:28:59.722448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:28:59.806809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687d5a-2d90-4be6-8072-96907a69ae3a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:28:59.811543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:28:59.812413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:28:59.812736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:28:59.813015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] PATH_INFO: `/v3/users/5b2ce2786157469f818ff9c8216433ef` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:28:59.814203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2ce2786157469f818ff9c8216433ef)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:28:59.816119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:00.099985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2ce2786157469f818ff9c8216433ef'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:00.106687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2ce2786157469f818ff9c8216433ef'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:00.107413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2ce2786157469f818ff9c8216433ef'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:00.194637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2ce2786157469f818ff9c8216433ef'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:00.195282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2ce2786157469f818ff9c8216433ef'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:00.219502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 876/1753] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:28:59 2022] DELETE /identity/v3/users/5b2ce2786157469f818ff9c8216433ef => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:00.224246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:00.224761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:00.224836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:00.244511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:00.245755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:00.266198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:00.307928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 878/1754] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:00.317695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:00.319366 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:00.378984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa4958-7f93-4043-bce9-3fe750f9b4c7 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:00.382594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:00.383242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:00.383528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:00.383809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:00.393219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:00.394678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:00.404610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 877/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:00.972831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:00.972831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:00.983260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673db2b-273c-4618-aa3f-00c374d14631 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:00.988165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:00.989137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:00.989419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:00.989697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] PATH_INFO: `/v3/projects/6f93ad57f88f410296f38953d870d002` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:00.990892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f93ad57f88f410296f38953d870d002)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:00.992826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:01.166077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f93ad57f88f410296f38953d870d002', 'request_id': 'req-a46aac25-80cd-4496-ab90-b6da137638e7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.174018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f93ad57f88f410296f38953d870d002', 'request_id': 'req-a46aac25-80cd-4496-ab90-b6da137638e7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.180757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f93ad57f88f410296f38953d870d002', 'request_id': 'req-a46aac25-80cd-4496-ab90-b6da137638e7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.315012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] The token cache is being invalidate because project 6f93ad57f88f410296f38953d870d002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:01.315571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.316127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.317891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 879/1756] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:00 2022] DELETE /identity/v3/projects/6f93ad57f88f410296f38953d870d002 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:01.321955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:01.332288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e1b0e-f02e-45f7-acbb-bd25a98dbce8 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:01.393805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:01.399971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:01.399971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:01.399971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] PATH_INFO: `/v3/users/f0988ee57e0b411ea7f1a023edfe508f` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:01.399971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0988ee57e0b411ea7f1a023edfe508f)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:01.399971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:01.750900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0988ee57e0b411ea7f1a023edfe508f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.757636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0988ee57e0b411ea7f1a023edfe508f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.758311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0988ee57e0b411ea7f1a023edfe508f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.869564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0988ee57e0b411ea7f1a023edfe508f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.870328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-05cf69c5-16a5-49dd-bc53-1a7b250e48f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0988ee57e0b411ea7f1a023edfe508f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:01.890671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 878/1757] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:01 2022] DELETE /identity/v3/users/f0988ee57e0b411ea7f1a023edfe508f => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:02.308848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:02.512995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:02.515272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:02.556410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aac25-80cd-4496-ab90-b6da137638e7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:02.616294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:02.635000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:02.635878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:02.636159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:02.636433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] PATH_INFO: `/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:02.637791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7ff5e8188e483c8b4ac61db2c3b388)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:02.640248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:02.752631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7ff5e8188e483c8b4ac61db2c3b388', 'request_id': 'req-8ba4b2e2-683e-44ae-a7f8-3d845c328763'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:02.759188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7ff5e8188e483c8b4ac61db2c3b388', 'request_id': 'req-8ba4b2e2-683e-44ae-a7f8-3d845c328763'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:02.764913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7ff5e8188e483c8b4ac61db2c3b388', 'request_id': 'req-8ba4b2e2-683e-44ae-a7f8-3d845c328763'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:02.869693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] The token cache is being invalidate because project 6a7ff5e8188e483c8b4ac61db2c3b388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:02.869965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:02.870596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8ba4b2e2-683e-44ae-a7f8-3d845c328763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:02.872150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 880/1758] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:02 2022] DELETE /identity/v3/projects/6a7ff5e8188e483c8b4ac61db2c3b388 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:02.877489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:02.877977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:02.878112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:02.897184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:02.898395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:02.956529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 879/1759] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:02.964569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:02.964871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:02.965150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:02.985066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:02.986311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:02.995056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 881/1760] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:02.998884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.009090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bb3a9-2ff9-433b-bd72-6709711a0a8f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.014015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.014730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.015018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.015295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.016032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:03.016339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.018029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.030894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 880/1761] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:03.035477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.046481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fe40-66f1-4984-a626-6d2da03515a2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.051512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.052600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.053027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.053303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.054001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.055911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.213895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 882/1762] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:03 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:03.217868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.227852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ffd0-57db-4b99-b295-c6f062138c61 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.284225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.284943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.285228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.285499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.286179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.288084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.408985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 881/1763] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:03 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:03.412956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.423247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ecc89-5a6b-457c-a852-215aaa9092f4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.427810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.428463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.428792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.429073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.429819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.431626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.442114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 883/1764] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:03.445866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.456354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70dde-e7f5-45a7-b81c-8aba3df92e51 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.460950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.461651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.461925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.462198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.462915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.464533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7d706-200e-4f75-8f10-c9bf1d7a7b0d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.475434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 882/1765] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:03.479508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.490498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dcbd-7ad6-4464-a575-0dd0645729d8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.495417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.496357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.496678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.496958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] PATH_INFO: `/v3/projects/0ea4e2754da54585a3796e6ba91d4237/users/e9f5e91a02354cd8ad6c3133682db17a/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.513258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea4e2754da54585a3796e6ba91d4237, user_id=e9f5e91a02354cd8ad6c3133682db17a, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.515897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:03.614310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 884/1766] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:03 2022] PUT /identity/v3/projects/0ea4e2754da54585a3796e6ba91d4237/users/e9f5e91a02354cd8ad6c3133682db17a/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:03.626173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.626458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.626732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.658298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] MFA Rules not processed for user `e9f5e91a02354cd8ad6c3133682db17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:03.659504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:03.724781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 883/1767] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:03.732660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:03.811729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49387852-5ee0-4cf8-94e7-7e9eb678354a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:03.816495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:03.817188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:03.817471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:03.817746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:03.818452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:03.820392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.020912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 885/1768] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:03 2022] POST /identity/v3/projects => generated 375 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:04.022904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.034413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ee6c0-0fd3-40be-979b-cf8de0bdbfc4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.054750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:04.098161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.098856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.107411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.107411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.107411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.107411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.309550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 884/1769] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:04 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:04.313502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.324098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109707e2-62a6-4bd9-8230-67f64a459aa2 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.328931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.329678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.329876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.330151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.331086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.332488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.345807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 886/1770] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:04.349986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.360859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699012f-57eb-4711-b001-c210454dea61 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.365658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.366635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.367011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.367354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] PATH_INFO: `/v3/projects/a4923b5e049c468caa5d9017debacdc4/users/32238d194cab485089e45f38b2780746/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.382507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4923b5e049c468caa5d9017debacdc4, user_id=32238d194cab485089e45f38b2780746, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.385218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.537852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:04.543551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 885/1771] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:04 2022] PUT /identity/v3/projects/a4923b5e049c468caa5d9017debacdc4/users/32238d194cab485089e45f38b2780746/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:04.547637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.557508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2335b-6461-40b3-88bb-f0a7addd37fb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.616421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.617125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.617415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.617686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.618557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.620136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301868c-57e7-42a9-8e3d-cf95ceb0b2b2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.630485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 887/1772] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:04.634941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.652260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bcffe-4b5f-4a49-b941-304114ca1306 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.653089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.654816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.655167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.655515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] PATH_INFO: `/v3/projects/a4923b5e049c468caa5d9017debacdc4/users/32238d194cab485089e45f38b2780746/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.657679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4923b5e049c468caa5d9017debacdc4, user_id=32238d194cab485089e45f38b2780746, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.660854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.685766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 886/1773] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:04 2022] PUT /identity/v3/projects/a4923b5e049c468caa5d9017debacdc4/users/32238d194cab485089e45f38b2780746/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:04.696427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f03ce-4e6d-4664-bce2-99f93c5bef5d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.696922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f03ce-4e6d-4664-bce2-99f93c5bef5d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.697142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f03ce-4e6d-4664-bce2-99f93c5bef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.724849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-844f03ce-4e6d-4664-bce2-99f93c5bef5d None None] MFA Rules not processed for user `32238d194cab485089e45f38b2780746`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:04.726287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-844f03ce-4e6d-4664-bce2-99f93c5bef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:04.798575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 888/1774] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:04.810133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:04.811924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:04.913520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094cbe-0886-4eb3-a690-71f341522551 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:04.918707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:04.918707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.918707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.918707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.971449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5993d-a0ec-4308-bf48-5689ff883f0d None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.971786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5993d-a0ec-4308-bf48-5689ff883f0d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.972286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5993d-a0ec-4308-bf48-5689ff883f0d None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:04.974888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 889/1775] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:29:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 09:29:04.988687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:04.989436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:04.991739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:04.992196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:04.992683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:05.018621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 887/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:05.039220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:05.040454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:05.112228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 890/1777] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:05.124805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:05.126252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:05.145524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de274ca-e489-4753-8d77-a13b4bec9d38 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:05.156318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:05.157151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:05.157561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:05.157914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:05.168882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:05.171348 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:05.188872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 888/1778] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:07.691569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:09.204166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:11.676604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:14.029172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:14.038635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af3210-000d-4c03-8a01-4012d9a0859e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:14.042238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:14.042869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:14.043147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:14.043420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:14.052640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:14.054019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b46a18-4718-4059-9f68-e863b31c4277 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:14.064941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 891/1779] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:14.148386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:15.064503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:15.066725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:15.183456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bbdcc-6f9c-4721-a244-ed90a9cdb6ef service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:15.188394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:15.189445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:15.189832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:15.190224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] PATH_INFO: `/v3/users/42315cae58e44bfdb6e7401dfbcd8688` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:15.191575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=42315cae58e44bfdb6e7401dfbcd8688)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:15.193663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:15.413395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42315cae58e44bfdb6e7401dfbcd8688'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:15.421599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42315cae58e44bfdb6e7401dfbcd8688'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:15.422592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42315cae58e44bfdb6e7401dfbcd8688'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:15.523305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:15.527484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42315cae58e44bfdb6e7401dfbcd8688'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:15.528470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42315cae58e44bfdb6e7401dfbcd8688'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:15.557757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 889/1780] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:15 2022] DELETE /identity/v3/users/42315cae58e44bfdb6e7401dfbcd8688 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:15.566219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:15.566716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:15.566802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:15.585046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:15.586270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:15.600108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:15.650379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 892/1781] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:15.660129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:15.661569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:15.718210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97a49e-a950-41a0-9d17-1cf1343308bf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:15.721932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:15.722573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:15.722860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:15.723147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:15.732182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:15.733910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:15.743331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 890/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:16.317886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:16.319722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:16.334984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21dacf-4d0e-406f-aca2-c5a7519bd380 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:16.340687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:16.341549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:16.341899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:16.342092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] PATH_INFO: `/v3/projects/2ff66bebe7064c9fa058deb52fdfcc31` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:16.343340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff66bebe7064c9fa058deb52fdfcc31)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:16.345196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:16.475296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff66bebe7064c9fa058deb52fdfcc31', 'request_id': 'req-57dd34c0-8f02-4490-8af2-a861a569363b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.481437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff66bebe7064c9fa058deb52fdfcc31', 'request_id': 'req-57dd34c0-8f02-4490-8af2-a861a569363b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.487193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff66bebe7064c9fa058deb52fdfcc31', 'request_id': 'req-57dd34c0-8f02-4490-8af2-a861a569363b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.579009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] The token cache is being invalidate because project 2ff66bebe7064c9fa058deb52fdfcc31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:16.579552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.579913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.581607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 893/1783] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:16 2022] DELETE /identity/v3/projects/2ff66bebe7064c9fa058deb52fdfcc31 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:16.585856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:16.596401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62a636-b675-42ae-bd10-2dc096d3d41c service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:16.655152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:16.656139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:16.656328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:16.656641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] PATH_INFO: `/v3/users/6908775194b741efbe14636c714cedca` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:16.658508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6908775194b741efbe14636c714cedca)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:16.660901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:16.768421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:16.924221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6908775194b741efbe14636c714cedca'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.931077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6908775194b741efbe14636c714cedca'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:16.931785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6908775194b741efbe14636c714cedca'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.059203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6908775194b741efbe14636c714cedca'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.060061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19d20fc7-a744-4c9a-80ba-bfe032ef1de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6908775194b741efbe14636c714cedca'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.082902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 891/1784] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:16 2022] DELETE /identity/v3/users/6908775194b741efbe14636c714cedca => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:17.557723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:17.559178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:17.573432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd34c0-8f02-4490-8af2-a861a569363b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:17.633118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:17.633984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:17.637898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:17.637898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] PATH_INFO: `/v3/projects/0cd554a672414f6d99872135eba8768c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:17.637898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd554a672414f6d99872135eba8768c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:17.637898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:17.753143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:17.755887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd554a672414f6d99872135eba8768c', 'request_id': 'req-a2ff5d86-4305-44f1-b15f-b225b64f751d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.762426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd554a672414f6d99872135eba8768c', 'request_id': 'req-a2ff5d86-4305-44f1-b15f-b225b64f751d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.767935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd554a672414f6d99872135eba8768c', 'request_id': 'req-a2ff5d86-4305-44f1-b15f-b225b64f751d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.870039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] The token cache is being invalidate because project 0cd554a672414f6d99872135eba8768c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:17.870417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.871171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a2ff5d86-4305-44f1-b15f-b225b64f751d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:17.873063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 894/1785] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:17 2022] DELETE /identity/v3/projects/0cd554a672414f6d99872135eba8768c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:17.877911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:17.878387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:17.878663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:17.896716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:17.898895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:17.957403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 892/1786] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:17.965129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:17.965418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:17.965695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:17.985012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:17.986152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:17.994488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 895/1787] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:17.998728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.008640 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab52372-a375-4658-94e8-2470e582c95d None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.013786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.013919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.014208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.014483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.015304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:18.015509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.017165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.030000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 893/1788] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:18.034384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.045379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7fd9-8067-4174-8cf1-aaac017940d2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.050068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.050708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.050986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.051255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.051978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.053875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.171425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 896/1789] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:18 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:18.172597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.184148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f58de6-89f8-406f-bda6-546cff097a6b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.264952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.266297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.266680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.267055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.268453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.270984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.362652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 894/1790] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:18 2022] POST /identity/v3/users => generated 282 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:18.367141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.378786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef235-15f1-47bf-b863-09e7cc8db849 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.383371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.384020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.384350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.384650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.385369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.386941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.401456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 897/1791] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:18.402674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.412956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc80d-280d-429a-b81a-1863faf22f7f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.417845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.418599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.419409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.419692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.420422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.422119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72848b3-7c64-43ec-af35-d2ded7ebacd0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.432283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 895/1792] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:18.437214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.447360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987245c-d3d0-4baa-90a1-ec35766e3e8e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.452005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.452914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.453193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.453466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] PATH_INFO: `/v3/projects/139fe1e9b426477faa53b6c53e215a05/users/4624ef37586e4442ac72fc493e14d934/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.469129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139fe1e9b426477faa53b6c53e215a05, user_id=4624ef37586e4442ac72fc493e14d934, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.471777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.596478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 898/1793] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:18 2022] PUT /identity/v3/projects/139fe1e9b426477faa53b6c53e215a05/users/4624ef37586e4442ac72fc493e14d934/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:18.607437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.607720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.607993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.627596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:18.636350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] MFA Rules not processed for user `4624ef37586e4442ac72fc493e14d934`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:18.637588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:18.699599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 896/1794] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:18.704849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.766717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653a510-ad54-4fd0-9d3d-8a5fdf8d4f06 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.771346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.771996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.772280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.772556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.773277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.775209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:18.859225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 899/1795] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:18 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:18.863302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:18.874241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4e14b-949b-4f5c-8615-ab1ca8668048 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:18.933703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:18.934381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:18.934682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:18.934957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:18.935653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:18.937676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.032962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 897/1796] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:18 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:19.034171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.045044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e42fdc-413f-4840-98dc-49f2cc26f9ae admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.049930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.050593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.050878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.051318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.057038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.058701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.070671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 900/1797] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:19.074692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.085807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b697-9de1-44af-9a9c-a639d0ae4979 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.090593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.092082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.092347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.092673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] PATH_INFO: `/v3/projects/42943a40e62d410d95bdc12f2a0bc30a/users/497e6b30c0eb4ef4b466c8861df349b0/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.107380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42943a40e62d410d95bdc12f2a0bc30a, user_id=497e6b30c0eb4ef4b466c8861df349b0, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.110428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.231043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 898/1798] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:19 2022] PUT /identity/v3/projects/42943a40e62d410d95bdc12f2a0bc30a/users/497e6b30c0eb4ef4b466c8861df349b0/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:19.235239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.245172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ded6-8680-4eb7-a3fc-0da77df9b99b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.286300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:19.301572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.302244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.302523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.302799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.303534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.305213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2489c-d8d3-4275-83d3-ffb87acbfa68 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.315981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 901/1799] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:19.320523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.332816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006ccd8-cf19-48ae-933e-169470fdb5c2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.339267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.340453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.341243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.341243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] PATH_INFO: `/v3/projects/42943a40e62d410d95bdc12f2a0bc30a/users/497e6b30c0eb4ef4b466c8861df349b0/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.343152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42943a40e62d410d95bdc12f2a0bc30a, user_id=497e6b30c0eb4ef4b466c8861df349b0, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.345936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.358281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 899/1800] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:19 2022] PUT /identity/v3/projects/42943a40e62d410d95bdc12f2a0bc30a/users/497e6b30c0eb4ef4b466c8861df349b0/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:19.368120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.368420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.368737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.406994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] MFA Rules not processed for user `497e6b30c0eb4ef4b466c8861df349b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:19.408455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:19.495285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 902/1801] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:19.507903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.509616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:19.568127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689be5-6c0b-44da-8e9d-4609a36d0289 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.571965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.572634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.572935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.573216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.580082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:19.583888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.585370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.595233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 900/1802] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:19.831167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:19.842282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f68a5-a1bf-4dc0-8300-26441c2d8e12 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:19.846327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:19.847174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:19.847598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:19.848013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:19.910370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:19.911742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ae8d-29ac-4b1a-a36e-67d208cf5d68 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:19.921696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 903/1803] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:20.976284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:20.977736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:21.043869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118718af-a6f2-49cd-891f-b9a865c9fd38 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:21.048472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:21.049367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:21.049650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:21.049921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] PATH_INFO: `/v3/users/7523b43aa7f54b3f9e92139486fad6da` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:21.051098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7523b43aa7f54b3f9e92139486fad6da)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:21.053047 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:21.307437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7523b43aa7f54b3f9e92139486fad6da'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:21.317051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7523b43aa7f54b3f9e92139486fad6da'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:21.317838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7523b43aa7f54b3f9e92139486fad6da'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:21.394116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7523b43aa7f54b3f9e92139486fad6da'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:21.395141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7523b43aa7f54b3f9e92139486fad6da'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:21.413045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 901/1804] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:20 2022] DELETE /identity/v3/users/7523b43aa7f54b3f9e92139486fad6da => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:21.417645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:21.418100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:21.418236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:21.436625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:21.437827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:21.509543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 904/1805] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:21.520866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:21.522783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:21.593107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560619-5ecb-4b47-87dd-7129e2eef87f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:21.597135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:21.597980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:21.598267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:21.598689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:21.608868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:21.610306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:21.621043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 902/1806] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:22.360177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:22.361696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:22.371523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:22.376671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8857570-02ba-4a6b-85ae-e81735159206 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:22.386045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:22.389265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:22.390256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:22.390698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] PATH_INFO: `/v3/projects/8b8587f3ac1e42a89a81d91c7a515d31` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:22.392469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8587f3ac1e42a89a81d91c7a515d31)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:22.394714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:22.592737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8587f3ac1e42a89a81d91c7a515d31', 'request_id': 'req-357fe337-7739-4b16-a456-2812880e7a23'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:22.599109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8587f3ac1e42a89a81d91c7a515d31', 'request_id': 'req-357fe337-7739-4b16-a456-2812880e7a23'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:22.607210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8587f3ac1e42a89a81d91c7a515d31', 'request_id': 'req-357fe337-7739-4b16-a456-2812880e7a23'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:22.731936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] The token cache is being invalidate because project 8b8587f3ac1e42a89a81d91c7a515d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:22.732370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:22.733066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:22.735783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 905/1807] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:22 2022] DELETE /identity/v3/projects/8b8587f3ac1e42a89a81d91c7a515d31 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:22.739892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:22.755128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a27165-de9a-4dc6-b8d0-2a9abe05c454 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:22.767414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:22.883887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:22.884729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:22.884729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:22.886205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] PATH_INFO: `/v3/users/54167db9f3484d5488a029bee34e4a84` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:22.887931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] RBAC: Authorizing `identity:delete_user(user_id=54167db9f3484d5488a029bee34e4a84)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:22.890269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:23.173745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54167db9f3484d5488a029bee34e4a84'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:23.184644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54167db9f3484d5488a029bee34e4a84'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:23.184644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54167db9f3484d5488a029bee34e4a84'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:23.322443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54167db9f3484d5488a029bee34e4a84'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:23.323079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54167db9f3484d5488a029bee34e4a84'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:23.342024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 903/1808] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:22 2022] DELETE /identity/v3/users/54167db9f3484d5488a029bee34e4a84 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:23.990597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:23.992032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.006575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fe337-7739-4b16-a456-2812880e7a23 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.075085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.075963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.076241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.076514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] PATH_INFO: `/v3/projects/23b032f9567a4dfa9d34599bb320d765` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.077821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b032f9567a4dfa9d34599bb320d765)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.079812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.241747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.244231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.248914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b032f9567a4dfa9d34599bb320d765', 'request_id': 'req-cba6b836-55fd-4c39-90c5-2e23679ad519'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.255484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b032f9567a4dfa9d34599bb320d765', 'request_id': 'req-cba6b836-55fd-4c39-90c5-2e23679ad519'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.260764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb785f8b-12ec-4e83-9032-96e92791214b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.265997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b032f9567a4dfa9d34599bb320d765', 'request_id': 'req-cba6b836-55fd-4c39-90c5-2e23679ad519'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.327984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.329007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.329354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.329721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] PATH_INFO: `/v3/users/4624ef37586e4442ac72fc493e14d934` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.330986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4624ef37586e4442ac72fc493e14d934)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.333059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.355363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] The token cache is being invalidate because project 23b032f9567a4dfa9d34599bb320d765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:24.355657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.356425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cba6b836-55fd-4c39-90c5-2e23679ad519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.358042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 906/1809] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:23 2022] DELETE /identity/v3/projects/23b032f9567a4dfa9d34599bb320d765 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:24.363015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bda99-41c7-4417-bf36-c04fad383cc6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.363457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bda99-41c7-4417-bf36-c04fad383cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.363736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bda99-41c7-4417-bf36-c04fad383cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.383515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a86bda99-41c7-4417-bf36-c04fad383cc6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:24.384840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a86bda99-41c7-4417-bf36-c04fad383cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.393485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 907/1810] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:24.401050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.401365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.401680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.421074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:24.422292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.433034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 908/1811] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:24.437270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.442818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:24.446523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec76a-1cd0-4091-b2a6-cb0c78a92b9f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.451658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.452291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.452571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.452870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.453607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:24.453885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.455509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.467440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 909/1812] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:29:24.471622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.480970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725939d-0e92-49c4-804e-d0e6a61e452d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.485749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.486358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.489515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.489515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.489515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.489515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.516380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:24.524387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4624ef37586e4442ac72fc493e14d934'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.531079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4624ef37586e4442ac72fc493e14d934'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.531727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4624ef37586e4442ac72fc493e14d934'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.586143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 910/1813] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:24 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:24.590141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.605739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4624ef37586e4442ac72fc493e14d934'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.606517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b56f6b37-ba54-48b2-a563-545cedae2424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4624ef37586e4442ac72fc493e14d934'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:24.624948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 904/1814] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:24 2022] DELETE /identity/v3/users/4624ef37586e4442ac72fc493e14d934 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:24.629262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.629662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.630010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.648528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:24.649689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.652511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634f3a-6607-4e17-92e7-01a0349b6cdb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.720393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 905/1815] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:24.720760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.721426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.721694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.721973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.722658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.724640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.728673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.729963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.790487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4eed8-80e1-4ec5-ace9-5bd69a665b84 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.794198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.794938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.795229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.795510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.805693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.807155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.818545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 906/1816] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:24.824869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 911/1817] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:24 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:24.829205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.830570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:24.843973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae319778-0bd6-4588-be88-27eeb12b76ba service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.848569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.849925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.850229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.850581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.851210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.852852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.875758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 907/1818] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:24.877089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.888611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063205b-b3c8-494c-bcd5-e43f93c7b8a3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.894108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.897912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.897912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.897912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.897912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.898255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6312098-f10a-4ec6-a1ea-16f4c6ce879a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:24.911986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 912/1819] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:24.914715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:24.924337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd679a7-59bb-476d-90f7-14dbf273b322 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:24.929067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:24.929946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:24.930228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:24.930622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] PATH_INFO: `/v3/projects/f27fd1fbf7d04c38852d3c3ed68a8d29/users/9185f50dd30c4c139ae2144d9b4835a6/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:24.947544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27fd1fbf7d04c38852d3c3ed68a8d29, user_id=9185f50dd30c4c139ae2144d9b4835a6, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:24.950333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.066019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 908/1820] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:24 2022] PUT /identity/v3/projects/f27fd1fbf7d04c38852d3c3ed68a8d29/users/9185f50dd30c4c139ae2144d9b4835a6/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:25.077673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.078180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.078291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.106885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] MFA Rules not processed for user `9185f50dd30c4c139ae2144d9b4835a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:25.108111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:25.155956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:25.177700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 913/1821] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:25.182407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.209031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:25.250273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c3ef-1a63-42b4-ac99-2e38efc5f5f9 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.255115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.256812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.256812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.256812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.257238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.259510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.357908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.359191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 909/1822] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:25 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:25.359803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:25.362861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.373934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1765-a66a-4881-b49a-99c88705bc7d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.433249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03dcd1-aeb4-44c0-b9a7-9f8f3f62d9c9 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.438378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.439046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.439352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.439628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.440492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.442478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.458258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.463160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.463160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.463160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] PATH_INFO: `/v3/projects/139fe1e9b426477faa53b6c53e215a05` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.463160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=139fe1e9b426477faa53b6c53e215a05)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.463160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.565019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 910/1823] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:25 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:25.566666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.575874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633452fe-f4b4-4e3d-a570-79e54547508b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.580478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.581154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.581434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.581708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.582428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.584022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.597461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 911/1824] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:25.604957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.613562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139fe1e9b426477faa53b6c53e215a05', 'request_id': 'req-d8a026ed-bfc8-4299-9be3-efd79708a8ed'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.616207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64cc5-67a0-481f-b739-ccd8cc2ef850 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.620153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139fe1e9b426477faa53b6c53e215a05', 'request_id': 'req-d8a026ed-bfc8-4299-9be3-efd79708a8ed'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.625771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139fe1e9b426477faa53b6c53e215a05', 'request_id': 'req-d8a026ed-bfc8-4299-9be3-efd79708a8ed'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.671633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.672515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.672824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.673097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] PATH_INFO: `/v3/projects/171a8dcf61e6436aaa5fd70964692932/users/ddf272a79cd74c89a5abe150ff9dfdc6/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.687717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171a8dcf61e6436aaa5fd70964692932, user_id=ddf272a79cd74c89a5abe150ff9dfdc6, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.690448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.707596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] The token cache is being invalidate because project 139fe1e9b426477faa53b6c53e215a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:25.710711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.710711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.710711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 914/1825] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:25 2022] DELETE /identity/v3/projects/139fe1e9b426477faa53b6c53e215a05 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:25.713807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.715090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:25.727379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a026ed-bfc8-4299-9be3-efd79708a8ed admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.732612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.733739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.734065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.734338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] PATH_INFO: `/v3/users/497e6b30c0eb4ef4b466c8861df349b0` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.735548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497e6b30c0eb4ef4b466c8861df349b0)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.737490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.805485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 912/1826] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:25 2022] PUT /identity/v3/projects/171a8dcf61e6436aaa5fd70964692932/users/ddf272a79cd74c89a5abe150ff9dfdc6/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:25.809324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.811485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:25.821468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:25.876355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6bb4-466d-47e5-8d74-16b33135d494 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.881388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.882060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.882347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.882627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.883391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.885070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.895966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 913/1827] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:25.899939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:25.911031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b573bd-cfe1-4996-a8a5-4464f7875128 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:25.916414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:25.918160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.920905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.920905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] PATH_INFO: `/v3/projects/171a8dcf61e6436aaa5fd70964692932/users/ddf272a79cd74c89a5abe150ff9dfdc6/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.921847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=171a8dcf61e6436aaa5fd70964692932, user_id=ddf272a79cd74c89a5abe150ff9dfdc6, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:25.921986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497e6b30c0eb4ef4b466c8861df349b0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.924744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74320e-1219-4760-b12d-0f1d56719b5d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:25.930526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497e6b30c0eb4ef4b466c8861df349b0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:25.936707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 914/1828] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:25 2022] PUT /identity/v3/projects/171a8dcf61e6436aaa5fd70964692932/users/ddf272a79cd74c89a5abe150ff9dfdc6/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:25.951689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:25.951977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:25.952255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:25.979406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] MFA Rules not processed for user `ddf272a79cd74c89a5abe150ff9dfdc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:25.983903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.001828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497e6b30c0eb4ef4b466c8861df349b0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.002603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497e6b30c0eb4ef4b466c8861df349b0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.003314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497e6b30c0eb4ef4b466c8861df349b0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.015327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:26.034526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 915/1829] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:25 2022] DELETE /identity/v3/users/497e6b30c0eb4ef4b466c8861df349b0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:26.062981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 915/1830] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:26.068134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.070149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.130388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d206e-bc11-43ef-b94a-2c8f13b08a28 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:26.134299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:26.135019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.135645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.135645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.201519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:26.202962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:26.213174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 916/1831] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:26.243946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.245829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.313077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7457af2-2969-4a51-b6e9-5c78cd2dfa76 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:26.336053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:26.376577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:26.377566 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.377947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.378288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] PATH_INFO: `/v3/users/9185f50dd30c4c139ae2144d9b4835a6` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.379818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9185f50dd30c4c139ae2144d9b4835a6)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:26.381876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:26.577589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.579063 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.592513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe306-b510-42be-be12-643f39bdc68e service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:26.597209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:26.598067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.598347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.598617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] PATH_INFO: `/v3/projects/42943a40e62d410d95bdc12f2a0bc30a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.599785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42943a40e62d410d95bdc12f2a0bc30a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:26.601693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:26.653201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9185f50dd30c4c139ae2144d9b4835a6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.660305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9185f50dd30c4c139ae2144d9b4835a6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.661055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9185f50dd30c4c139ae2144d9b4835a6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.741823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42943a40e62d410d95bdc12f2a0bc30a', 'request_id': 'req-339c9f1f-aa58-42ad-8836-fbba24fe8694'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.748544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42943a40e62d410d95bdc12f2a0bc30a', 'request_id': 'req-339c9f1f-aa58-42ad-8836-fbba24fe8694'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.755112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42943a40e62d410d95bdc12f2a0bc30a', 'request_id': 'req-339c9f1f-aa58-42ad-8836-fbba24fe8694'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.756400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9185f50dd30c4c139ae2144d9b4835a6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.759315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1bdfb11d-2ade-4269-9fe7-9f5d8785b509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9185f50dd30c4c139ae2144d9b4835a6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.776272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 916/1832] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:26 2022] DELETE /identity/v3/users/9185f50dd30c4c139ae2144d9b4835a6 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:26.781135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.781454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.781769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.798734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:26.799931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.859057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] The token cache is being invalidate because project 42943a40e62d410d95bdc12f2a0bc30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:26.859699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.861251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-339c9f1f-aa58-42ad-8836-fbba24fe8694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:26.863506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 917/1833] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:26 2022] DELETE /identity/v3/projects/42943a40e62d410d95bdc12f2a0bc30a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:26.867490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 917/1834] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:26.873576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347ba8-4150-4d9b-a4f0-2a2977f6a3fc None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.873885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347ba8-4150-4d9b-a4f0-2a2977f6a3fc None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.874173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347ba8-4150-4d9b-a4f0-2a2977f6a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.875294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.876906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.896283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-28347ba8-4150-4d9b-a4f0-2a2977f6a3fc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:26.898074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-28347ba8-4150-4d9b-a4f0-2a2977f6a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.908356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 918/1835] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:26.915750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.916135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.916413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.929019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:26.936421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:26.940438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.941230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8cab1-6ec5-4b45-8a07-fd45bf13a896 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:26.944935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:26.945642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.945922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.946195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.947133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 919/1836] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:26.948128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:26.950850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.959806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed84379-29df-4ca5-b021-eb4e52c5ed72 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:26.960890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:26.962321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:26.964483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:26.965166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:26.965479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:26.966445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:26.967097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:26.967424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:26.969032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:26.971504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:26.973454 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 918/1837] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:26.983193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 920/1838] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:29:26.987514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:26.998616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823324-1656-436d-884a-5b9ab53090d1 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.018140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.018834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.019339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.019495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.020097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.026093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.163450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 919/1839] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:26 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:27.167765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.183755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f64a5-32f1-4bfe-9b98-7f4cf95e5563 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.238612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.239452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.239842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.240294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.241209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.243210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.348069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 921/1840] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:27 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:27.352099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.363146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255680f-069f-43b4-8b69-6f432cbb457e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.368252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.369040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.369394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.369743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.370596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.372308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.386917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 920/1841] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:27.388814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.399404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07531437-fde9-4a62-91c9-740c242bd4bd admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.405009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.405518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.405849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.415436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.416282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.418573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.430388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 922/1842] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:27.433446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.443679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def9f1-0a5d-481e-9746-20e3eca3d5bd admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.448711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.449596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.450578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.450853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] PATH_INFO: `/v3/projects/f009a7a071f146bf8115da4fcf328e2a/users/d1bac82c0396416aa875baffb06d3268/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.467635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f009a7a071f146bf8115da4fcf328e2a, user_id=d1bac82c0396416aa875baffb06d3268, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.470334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da221c13-5b59-4017-9c36-867bea721ed9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.558646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 921/1843] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:27 2022] PUT /identity/v3/projects/f009a7a071f146bf8115da4fcf328e2a/users/d1bac82c0396416aa875baffb06d3268/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:27.558937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.558937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:27.562503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.562687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.562963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.579465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13471f2-0403-4074-9d2d-aa0f0c1e0a4d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.603126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] MFA Rules not processed for user `d1bac82c0396416aa875baffb06d3268`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:27.604773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:27.642737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.644179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.644462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.644780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] PATH_INFO: `/v3/projects/f27fd1fbf7d04c38852d3c3ed68a8d29` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.646004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27fd1fbf7d04c38852d3c3ed68a8d29)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.648268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.679952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 922/1844] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:27.695304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.705620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714a1ff-6eac-4763-af3c-47c6c982f209 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.710887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.711554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.711823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.712096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.712820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.715027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.808689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27fd1fbf7d04c38852d3c3ed68a8d29', 'request_id': 'req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:27.814942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27fd1fbf7d04c38852d3c3ed68a8d29', 'request_id': 'req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:27.820703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27fd1fbf7d04c38852d3c3ed68a8d29', 'request_id': 'req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:27.872031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 923/1845] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:27 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:27.876190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.894061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] The token cache is being invalidate because project f27fd1fbf7d04c38852d3c3ed68a8d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:27.894253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:27.895104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:27.898516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 923/1846] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:27 2022] DELETE /identity/v3/projects/f27fd1fbf7d04c38852d3c3ed68a8d29 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:27.902525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:27.905081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:27.911148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:27.937382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:27.947763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f441e2-e7e3-47b4-8a52-df3dfe2d1001 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.952406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.953194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.953475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.953748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.954430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.956377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:27.976066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c990eb-b31a-4f86-86a6-a1cd3f15eb4c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:27.980738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:27.981629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:27.982238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:27.982238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] PATH_INFO: `/v3/users/ddf272a79cd74c89a5abe150ff9dfdc6` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:27.983618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf272a79cd74c89a5abe150ff9dfdc6)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:27.985560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.036200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 924/1847] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:27 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:28.039933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.041516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:28.053230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ff356-4789-45b2-a0bc-d26e7c7e7a5f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.057751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.058378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.058658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.058934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.059682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.061298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.071631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 925/1848] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:28.075932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.085084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ae64-02e0-487e-b602-f92d8505d670 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.089918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.090540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.090883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.091152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.091840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.093395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.102859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 926/1849] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:28.107165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.116782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345224d-d95e-4b3f-bd31-7ea3820d7453 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.121476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.122999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.123277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.125849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] PATH_INFO: `/v3/projects/65cb083306cd43e2b894f478cd1582eb/users/4b281101a58d4ff99242d70f27b3e572/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.137881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cb083306cd43e2b894f478cd1582eb, user_id=4b281101a58d4ff99242d70f27b3e572, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.140377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1a79c-b6d9-49a0-bbee-90ea41bf899c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.217553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf272a79cd74c89a5abe150ff9dfdc6'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:28.219544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 927/1850] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:28 2022] PUT /identity/v3/projects/65cb083306cd43e2b894f478cd1582eb/users/4b281101a58d4ff99242d70f27b3e572/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:28.224860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf272a79cd74c89a5abe150ff9dfdc6'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:28.229463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.229759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.230127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.258452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] MFA Rules not processed for user `4b281101a58d4ff99242d70f27b3e572`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:28.259770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:28.329700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 928/1851] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:28.333668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.342824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf272a79cd74c89a5abe150ff9dfdc6'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:28.343532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf272a79cd74c89a5abe150ff9dfdc6'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:28.344665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf272a79cd74c89a5abe150ff9dfdc6'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:28.365948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 924/1852] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:27 2022] DELETE /identity/v3/users/ddf272a79cd74c89a5abe150ff9dfdc6 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:28.409579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa65e4-2519-4cf0-a11e-10cbbee50148 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.451416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:28.476704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.477376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.477659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.477934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.478626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.480596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.571921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 929/1853] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:28 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:28.573116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.574728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:28.590539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab813-b0ab-46eb-ba34-53a13ffd41f6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.660302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.661069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.661357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.661633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.662340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.664403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.769062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 925/1854] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:28 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:28.773062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.785994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5594-39c4-475a-aa54-43231fb7932f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.791879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.793000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.797729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.797729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.797729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.797729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.808998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 930/1855] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:28.813933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:28.824753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef956-ff4d-4c3a-a214-60bfc610eb54 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:28.829746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:28.832109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:28.832109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:28.832109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] PATH_INFO: `/v3/projects/b6048dcd33a3438a82a7cc93a564160b/users/167d9a7415484fc1aae152c891c96764/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:28.841490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:28.849278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6048dcd33a3438a82a7cc93a564160b, user_id=167d9a7415484fc1aae152c891c96764, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:28.851976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:28.984231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 926/1856] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:28 2022] PUT /identity/v3/projects/b6048dcd33a3438a82a7cc93a564160b/users/167d9a7415484fc1aae152c891c96764/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:28.988567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.000349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a33a4d-c4f4-4b53-972d-2b6d8860500d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.070211 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.070881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.071167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.071444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.072168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.073869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d90d5-951f-438f-a4e6-e2e76d07ba3d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.083991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 931/1857] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:29.088261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.099489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d3011-fd31-47ca-b9c9-3fdd93ed1d3b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.104439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.105153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.105439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.105712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] PATH_INFO: `/v3/projects/b6048dcd33a3438a82a7cc93a564160b/users/167d9a7415484fc1aae152c891c96764/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.108505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6048dcd33a3438a82a7cc93a564160b, user_id=167d9a7415484fc1aae152c891c96764, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.111165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.121712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 927/1858] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:29 2022] PUT /identity/v3/projects/b6048dcd33a3438a82a7cc93a564160b/users/167d9a7415484fc1aae152c891c96764/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:29.133899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.134264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.134528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.141657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.144050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:29.167906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] MFA Rules not processed for user `167d9a7415484fc1aae152c891c96764`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:29.168681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:29.213813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a38bf-cc0b-4b27-918b-f44896f271f3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.219899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.220983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.221341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.221685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] PATH_INFO: `/v3/projects/171a8dcf61e6436aaa5fd70964692932` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.222940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171a8dcf61e6436aaa5fd70964692932)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.224938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.240382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 932/1859] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:29.251444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.252842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:29.287306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:29.308847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a6fa-cbd7-4a26-90df-bc763219f8a6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.313842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.314499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.314783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.315991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.324781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.327979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4774e9-fa87-43cd-8dc6-f1759bae2a79 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.338384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 933/1860] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:29.354314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171a8dcf61e6436aaa5fd70964692932', 'request_id': 'req-26fa2d39-82db-4d49-90e1-ac866d5c7272'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:29.375520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171a8dcf61e6436aaa5fd70964692932', 'request_id': 'req-26fa2d39-82db-4d49-90e1-ac866d5c7272'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:29.410801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171a8dcf61e6436aaa5fd70964692932', 'request_id': 'req-26fa2d39-82db-4d49-90e1-ac866d5c7272'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:29.505295 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] The token cache is being invalidate because project 171a8dcf61e6436aaa5fd70964692932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:29.505590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:29.506254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-26fa2d39-82db-4d49-90e1-ac866d5c7272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:29.508040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 928/1861] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:29 2022] DELETE /identity/v3/projects/171a8dcf61e6436aaa5fd70964692932 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:29.513382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.513770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.513952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.536563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:29.536738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:29.613039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 934/1862] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:29.620848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.621136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.621570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.689321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:29.690569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:29.703948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 929/1863] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:29.709355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.728406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0ddd-eb6a-4435-9738-c3dcd4481645 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.735562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.736328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.736644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.737408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.738411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:29.738731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.740931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.757713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 935/1864] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:29.762387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.772253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d7283-3be5-43f9-9eb5-d550c17d182f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.777416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:29.778659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:29.778945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:29.779216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:29.779942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:29.781961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:29.919216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 930/1865] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:29.922286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:29.932620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be916-c02d-4955-8af6-793530af8f78 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:29.997568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.000473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.000473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.000473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.000473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.001465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.117697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 936/1866] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:29 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:30.122187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.133536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb540284-bc7b-49f4-a123-dc6a4f64bbf7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.151177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.151896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.152176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.152447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.153253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.155035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.167546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 931/1867] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:30.182827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.195463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d58b0-739d-4a50-b518-5144d32e355d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.201394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.202261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.202670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.203010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.203857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.205749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c279-e93a-410c-8e04-14507b943abd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.221243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 937/1868] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:30.222625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.236128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41cb5-6dfe-4a79-8ed2-5160f79a5d10 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.244625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.244625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.244625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.245098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] PATH_INFO: `/v3/projects/4c03b9f02ad0400295ba775e7d075374/users/1b7e651f63c045e9be006d3fc85e659c/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.263329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c03b9f02ad0400295ba775e7d075374, user_id=1b7e651f63c045e9be006d3fc85e659c, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.266396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.368476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:30.372225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 932/1869] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:30 2022] PUT /identity/v3/projects/4c03b9f02ad0400295ba775e7d075374/users/1b7e651f63c045e9be006d3fc85e659c/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:30.383776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.384153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.384534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.419210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] MFA Rules not processed for user `1b7e651f63c045e9be006d3fc85e659c`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:30.420422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:30.452013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:30.487391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 938/1870] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:30.492004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.552572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610938bd-4c00-4a2d-9a1a-513349f251ec admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.557413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.558058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.558337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.558610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.559814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.561776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.681006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 933/1871] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:30 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:30.682106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.699251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc16d7a-5515-41d6-b097-58f4aeef7eec None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.755002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.756680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:30.757167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.757845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.758130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.758405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.759106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.761191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.817833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38a4-0bf7-4884-a31b-9a73ca70e325 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.821589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.822241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.822535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.822841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.824688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:30.867081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 939/1872] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:30 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:30.877776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.903731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.903731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.904138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b3b0-5f75-4587-ae50-bf9ea2e0f528 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.907154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.908052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.908234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 934/1873] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:30.908569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.909005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.910290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.911567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:30.923144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 940/1874] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:30.927248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:30.938983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f204-573d-4349-b8c1-5fb2ddfbfcd9 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:30.943906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:30.944832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:30.945118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:30.945529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] PATH_INFO: `/v3/projects/285499941d154d418605605bb686bf25/users/ed46ad0fa056479b93b8ad31e4624ad9/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:30.966266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285499941d154d418605605bb686bf25, user_id=ed46ad0fa056479b93b8ad31e4624ad9, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:30.967862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:31.044169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 935/1875] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:30 2022] PUT /identity/v3/projects/285499941d154d418605605bb686bf25/users/ed46ad0fa056479b93b8ad31e4624ad9/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:31.047686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:31.058538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb0833-f7ac-4308-9cb4-8998176dc9d5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:31.120904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:31.121568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:31.121879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:31.122155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:31.122876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:31.124517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc31d3-771f-48f7-83e5-a54354f9efc4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:31.134109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 941/1876] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:31.139126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:31.149573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3171f-3a57-4bfe-8909-9bcbc6aced05 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:31.154985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:31.155840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:31.156124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:31.156400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] PATH_INFO: `/v3/projects/285499941d154d418605605bb686bf25/users/ed46ad0fa056479b93b8ad31e4624ad9/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:31.158520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285499941d154d418605605bb686bf25, user_id=ed46ad0fa056479b93b8ad31e4624ad9, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:31.161242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:31.178990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 936/1877] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:31 2022] PUT /identity/v3/projects/285499941d154d418605605bb686bf25/users/ed46ad0fa056479b93b8ad31e4624ad9/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:31.189231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:31.189625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:31.189970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:31.255964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] MFA Rules not processed for user `ed46ad0fa056479b93b8ad31e4624ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:31.257508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:31.315910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:31.358650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 942/1878] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:31.633224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:31.636280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:31.644067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:31.702281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9837-0599-4d41-ba4c-46844f8cf154 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:31.705966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:31.706610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:31.707553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:31.707837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:31.717573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:31.719033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de06e6-044b-4908-8417-83439a20ad8d service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:31.728372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 937/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:31.849878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:31.915343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8f3d4-4991-44c2-ae49-fff080e269e5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:31.920483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:31.921417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:31.921706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:31.921982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] PATH_INFO: `/v3/users/1b7e651f63c045e9be006d3fc85e659c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:31.923207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7e651f63c045e9be006d3fc85e659c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:31.925212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:32.163396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7e651f63c045e9be006d3fc85e659c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:32.171479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7e651f63c045e9be006d3fc85e659c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:32.172163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7e651f63c045e9be006d3fc85e659c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:32.253018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7e651f63c045e9be006d3fc85e659c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:32.253694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7e651f63c045e9be006d3fc85e659c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:32.271729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 943/1880] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:31 2022] DELETE /identity/v3/users/1b7e651f63c045e9be006d3fc85e659c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:32.276253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:32.276548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:32.276868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:32.295855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:32.297125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:32.357071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 938/1881] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:32.366905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:32.368302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:32.426742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def8f4f-37aa-4e67-95fd-b7d2a69d90ff admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:32.430422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:32.431067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:32.431349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:32.431622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:32.437797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:32.441549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:32.442987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:32.453000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 944/1882] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:32.932628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:32.934131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:32.946219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b6348-6a7d-4d08-bdad-4ee8a39ffb2e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:32.950865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:32.951712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:32.951996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:32.952271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] PATH_INFO: `/v3/projects/4c03b9f02ad0400295ba775e7d075374` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:32.953501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c03b9f02ad0400295ba775e7d075374)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:32.955406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:33.067725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c03b9f02ad0400295ba775e7d075374', 'request_id': 'req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.074520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c03b9f02ad0400295ba775e7d075374', 'request_id': 'req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.080925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c03b9f02ad0400295ba775e7d075374', 'request_id': 'req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.178311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] The token cache is being invalidate because project 4c03b9f02ad0400295ba775e7d075374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:33.179397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.179397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.180877 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 939/1883] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:32 2022] DELETE /identity/v3/projects/4c03b9f02ad0400295ba775e7d075374 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:33.185016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:33.194818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6f6f-6ab0-4859-a13d-409b554bcd4f service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:33.253407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:33.254286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:33.254568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:33.254842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] PATH_INFO: `/v3/users/ed46ad0fa056479b93b8ad31e4624ad9` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:33.256098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed46ad0fa056479b93b8ad31e4624ad9)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:33.258064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:33.475983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed46ad0fa056479b93b8ad31e4624ad9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.483923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed46ad0fa056479b93b8ad31e4624ad9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.484602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed46ad0fa056479b93b8ad31e4624ad9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.582204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed46ad0fa056479b93b8ad31e4624ad9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.582840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ba9cf758-cc4e-4534-9cae-736c93f7b242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed46ad0fa056479b93b8ad31e4624ad9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:33.602277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 945/1884] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:33 2022] DELETE /identity/v3/users/ed46ad0fa056479b93b8ad31e4624ad9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:34.093026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.093026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:34.102519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc5cd5-b9df-4514-9adc-a1b379a5bb1b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:34.132997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:34.162450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:34.163307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.163593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.163870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] PATH_INFO: `/v3/projects/285499941d154d418605605bb686bf25` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.165079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=285499941d154d418605605bb686bf25)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:34.167040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:34.299307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285499941d154d418605605bb686bf25', 'request_id': 'req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:34.305439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285499941d154d418605605bb686bf25', 'request_id': 'req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:34.311559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285499941d154d418605605bb686bf25', 'request_id': 'req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:34.401184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] The token cache is being invalidate because project 285499941d154d418605605bb686bf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:34.401560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:34.402296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ef7f8d4a-e739-43f7-aa86-f1bbb060b4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:34.404680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 940/1885] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:34 2022] DELETE /identity/v3/projects/285499941d154d418605605bb686bf25 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:34.422304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.422771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.422988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.440507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:34.441826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:34.503951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 946/1886] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:34.511532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.511822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.512141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.531154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:34.532382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:34.541197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 941/1887] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:34.545674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.555350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7028573-db20-4e83-ba2c-917c4bfc3e41 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:34.560010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:34.560696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.560984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.561261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.561985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:34.562289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:34.563879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:34.572727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:34.576162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 947/1888] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:34.580152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.589451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6f2a7-aa67-4402-b630-7a848688f1ce None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:34.594581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:34.595256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.595533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.595801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.596494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:34.598411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:34.734734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 942/1889] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:34 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:34.738957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.753359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f178f-5b99-479e-9a5b-68e5b98a6aa3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:34.824009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:34.825398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.825746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.826059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.826664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:34.829034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:34.948288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 948/1890] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:34 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:34.949570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.960424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3573b-9c5e-4e21-ab9c-d12e7cef99ae admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:34.965123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:34.966399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:34.966687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:34.967035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:34.967821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:34.969420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:34.984283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 943/1891] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:34.987206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:34.998386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf254a-fe87-45c5-85bf-adc95398eaca admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.003777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.004452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.004801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.005244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.005986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.007634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee601-8713-4c05-a4bb-1e7de4b826d7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.020995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 949/1892] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:35.027525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.037614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f87f4-d17c-4e6c-97a7-b0eb537cd5d0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.042310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.043183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.043493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.044236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] PATH_INFO: `/v3/projects/8288e9b19ac340e484311ca19d0a733d/users/c49b76ad6b4a4bc8a87f40a5c19f276b/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.058982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8288e9b19ac340e484311ca19d0a733d, user_id=c49b76ad6b4a4bc8a87f40a5c19f276b, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.061989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.141040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 944/1893] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:35 2022] PUT /identity/v3/projects/8288e9b19ac340e484311ca19d0a733d/users/c49b76ad6b4a4bc8a87f40a5c19f276b/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:35.152437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.152759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.153064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.181109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] MFA Rules not processed for user `c49b76ad6b4a4bc8a87f40a5c19f276b`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:35.188749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:35.252490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 950/1894] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:35.256915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.314840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:35.319919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8828-298c-44ed-a960-4d39bd15d008 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.324557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.325852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.326127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.326398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.327078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.329070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.463328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 945/1895] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:35 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:35.464510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.475736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fb265-862d-4011-8468-33e6dbbe560f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.535179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.535861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.536109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.536379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.537100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.539054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.630946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:35.639532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 951/1896] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:35 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:35.640844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.651470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b9757-eb74-4dd3-84c8-de8329bdf393 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.656222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.656930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.657630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.657630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.658187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.664937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.676941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 946/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:35.681111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.691858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30f32c-098d-4e36-9655-9b1f2c34bca3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.697264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.698144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.698427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.698703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] PATH_INFO: `/v3/projects/cf4b3b2a61254675a04107e28d65389d/users/a00c2b4e939b4654a6ff5037a362e1ca/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.714727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4b3b2a61254675a04107e28d65389d, user_id=a00c2b4e939b4654a6ff5037a362e1ca, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.717466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.825007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 952/1898] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:35 2022] PUT /identity/v3/projects/cf4b3b2a61254675a04107e28d65389d/users/a00c2b4e939b4654a6ff5037a362e1ca/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:35.828799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.838756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fc377-4e9c-4910-997f-4aa4e4dc7d4a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.897611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.898291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.898546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.898821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.899555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.901240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341099-e64f-4d92-bf59-c77fd526c281 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.914357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 947/1899] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:35.916966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:35.928005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac9bff-ce09-4687-a29c-e734b503e3c1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:35.932747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:35.933626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.933905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.934221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] PATH_INFO: `/v3/projects/cf4b3b2a61254675a04107e28d65389d/users/a00c2b4e939b4654a6ff5037a362e1ca/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.936208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4b3b2a61254675a04107e28d65389d, user_id=a00c2b4e939b4654a6ff5037a362e1ca, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:35.938909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:35.949708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 953/1900] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:35 2022] PUT /identity/v3/projects/cf4b3b2a61254675a04107e28d65389d/users/a00c2b4e939b4654a6ff5037a362e1ca/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:35.959581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:35.959870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:35.960149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:35.987844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] MFA Rules not processed for user `a00c2b4e939b4654a6ff5037a362e1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:35.989070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:36.064017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 948/1901] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:36.080350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:36.083468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:36.172198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de59b89-c6b8-412c-a150-d969d582faf0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:36.176774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:36.177638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:36.177961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:36.178196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:36.189191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:36.190637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680fe05-9a0d-4f32-bdfc-22a812f4d37d service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:36.201392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 954/1902] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:36.774146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:36.833105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:36.846719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642c3d-a4d9-41be-91f5-3c4636f1d9ed None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:36.851960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:36.852873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:36.853164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:36.853469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] PATH_INFO: `/v3/users/c49b76ad6b4a4bc8a87f40a5c19f276b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:36.854665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49b76ad6b4a4bc8a87f40a5c19f276b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:36.856756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:37.078710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49b76ad6b4a4bc8a87f40a5c19f276b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:37.085697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49b76ad6b4a4bc8a87f40a5c19f276b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:37.182728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49b76ad6b4a4bc8a87f40a5c19f276b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:37.183439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49b76ad6b4a4bc8a87f40a5c19f276b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:37.184114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49b76ad6b4a4bc8a87f40a5c19f276b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:37.204042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 949/1903] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:36 2022] DELETE /identity/v3/users/c49b76ad6b4a4bc8a87f40a5c19f276b => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:37.208516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:37.209048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:37.209139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:37.229943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:37.231156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:37.297719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 955/1904] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:37.307079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:37.308562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:37.371425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3bbb6-d931-42fe-91ce-9cbba2064771 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:37.375674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:37.376427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:37.377165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:37.377165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:37.386716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:37.388293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:37.398330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 950/1905] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:37.846353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:37.849085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:37.863431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a128-ebc4-4a27-ab33-e0858e7c1304 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:37.869100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:37.873474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:37.873757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:37.874458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] PATH_INFO: `/v3/projects/8288e9b19ac340e484311ca19d0a733d` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:37.875518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8288e9b19ac340e484311ca19d0a733d)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:37.877492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:38.032628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8288e9b19ac340e484311ca19d0a733d', 'request_id': 'req-7681bcc4-eae3-4fb2-844c-09be52b6723e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.040011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8288e9b19ac340e484311ca19d0a733d', 'request_id': 'req-7681bcc4-eae3-4fb2-844c-09be52b6723e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.041647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:38.047914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8288e9b19ac340e484311ca19d0a733d', 'request_id': 'req-7681bcc4-eae3-4fb2-844c-09be52b6723e'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.201048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] The token cache is being invalidate because project 8288e9b19ac340e484311ca19d0a733d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:38.201342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.201959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.203507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 956/1906] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:37 2022] DELETE /identity/v3/projects/8288e9b19ac340e484311ca19d0a733d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:38.207622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:38.218353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369c47-caab-4446-8d44-af4ac1bfc9a3 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:38.288419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:38.289333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:38.289821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:38.290106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] PATH_INFO: `/v3/users/a00c2b4e939b4654a6ff5037a362e1ca` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:38.291368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00c2b4e939b4654a6ff5037a362e1ca)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:38.293336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:38.310321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:38.535301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00c2b4e939b4654a6ff5037a362e1ca'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.542215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00c2b4e939b4654a6ff5037a362e1ca'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.653713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00c2b4e939b4654a6ff5037a362e1ca'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.654337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00c2b4e939b4654a6ff5037a362e1ca'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.654950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-57122927-3f22-4220-b95a-16ac1a0a270a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00c2b4e939b4654a6ff5037a362e1ca'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:38.672409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 951/1907] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:38 2022] DELETE /identity/v3/users/a00c2b4e939b4654a6ff5037a362e1ca => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:39.225956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:39.228523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:39.246372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681bcc4-eae3-4fb2-844c-09be52b6723e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:39.321417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:39.322391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:39.322737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:39.323074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] PATH_INFO: `/v3/projects/cf4b3b2a61254675a04107e28d65389d` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:39.324355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4b3b2a61254675a04107e28d65389d)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:39.326861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:39.510075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4b3b2a61254675a04107e28d65389d', 'request_id': 'req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:39.522909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4b3b2a61254675a04107e28d65389d', 'request_id': 'req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:39.532980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4b3b2a61254675a04107e28d65389d', 'request_id': 'req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:39.624053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] The token cache is being invalidate because project cf4b3b2a61254675a04107e28d65389d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:39.624354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:39.625338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-80df854f-f1fb-4ff5-a3c4-7ec5dad092bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:39.627761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 957/1908] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:39 2022] DELETE /identity/v3/projects/cf4b3b2a61254675a04107e28d65389d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:39.632905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:39.633355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:39.633518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:39.653410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:39.654679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:39.725367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 952/1909] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:39.735363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:39.735650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:39.736185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:39.768262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:39.768262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:39.779268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 958/1910] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:39.783393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:39.795869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65246872-402b-4534-8495-b4b34d791fe0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:39.801406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:39.802137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:39.802429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:39.802781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:39.808482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:39.808482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:39.808482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:39.820394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 953/1911] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:39.824845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:39.835489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ede3a-9b02-4b0d-ac9e-d1b80a6f5f37 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:39.842894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:39.842894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:39.842894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:39.842894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:39.843898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:39.845946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.049755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 959/1912] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:40.050283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.062363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142115b1-1b3f-4bed-8de7-db1f779232fd admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.141895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.142506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.142771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.143044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.143741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.146075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.288967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 954/1913] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:40 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:40.294139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.305969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14a115-0cb2-4da1-906f-b9b9f83936e1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.311067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.312106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.312106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.312678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.313463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.315422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.328537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 960/1914] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:40.332618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.343949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96055-35f5-4105-ab60-235ada19c8ea admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.348677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.349371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.349654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.350034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.350762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.352347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc84e28-fe6f-436b-865b-808d313f675a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.354804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:40.365337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 955/1915] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:40.371013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.381534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83c55e-9d6b-4198-8a44-88f788b3b5b5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.386234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.387131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.387415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.387690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] PATH_INFO: `/v3/projects/02ecfb2cfd5e4ed2a2c78e66a1390116/users/6e6087e88e7a4e2f8866cb29e255c111/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.402567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ecfb2cfd5e4ed2a2c78e66a1390116, user_id=6e6087e88e7a4e2f8866cb29e255c111, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.405767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.489777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 961/1916] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:40 2022] PUT /identity/v3/projects/02ecfb2cfd5e4ed2a2c78e66a1390116/users/6e6087e88e7a4e2f8866cb29e255c111/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:40.500297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.500607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.500885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.530934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] MFA Rules not processed for user `6e6087e88e7a4e2f8866cb29e255c111`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:40.532300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:40.614864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 956/1917] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:40.626267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.628371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:40.647289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:40.697029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7be9-8085-48da-bbb5-b065c2850678 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.701378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.702121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.702473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.702852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.711606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.713164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77a0f-6548-4257-b1d2-9fe43fdbf105 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:40.722575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 962/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:40.742224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:40.812135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cd36b-fbf5-4d54-90be-d13c5e55ff98 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:40.816849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:40.817872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:40.818156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:40.818434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] PATH_INFO: `/v3/users/6e6087e88e7a4e2f8866cb29e255c111` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:40.819655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6087e88e7a4e2f8866cb29e255c111)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:40.821613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:41.082057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6087e88e7a4e2f8866cb29e255c111'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:41.091590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6087e88e7a4e2f8866cb29e255c111'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:41.180332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6087e88e7a4e2f8866cb29e255c111'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:41.181217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6087e88e7a4e2f8866cb29e255c111'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:41.181904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6087e88e7a4e2f8866cb29e255c111'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:41.218746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 957/1919] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:40 2022] DELETE /identity/v3/users/6e6087e88e7a4e2f8866cb29e255c111 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:41.226082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:41.226478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:41.226989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:41.251255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:41.252452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:41.312892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 963/1920] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:41.322678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:41.324446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:41.335676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:41.410049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90b174-249c-49fc-847c-c24dab1e52cb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:41.414675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:41.415360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:41.415642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:41.415916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:41.430239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:41.430239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bd36c-06f7-4d65-add6-f2ab9a40105e service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:41.440770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 958/1921] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:41.952055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:41.953920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:41.984376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca1c99-88ea-4643-a202-1abf5c05e8d9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:41.991466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:41.993162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:41.993444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:41.993749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] PATH_INFO: `/v3/projects/02ecfb2cfd5e4ed2a2c78e66a1390116` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:41.995226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ecfb2cfd5e4ed2a2c78e66a1390116)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:41.997974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.143749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ecfb2cfd5e4ed2a2c78e66a1390116', 'request_id': 'req-792dbee4-7008-44e8-b82e-4d57a3852d3c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:42.150198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ecfb2cfd5e4ed2a2c78e66a1390116', 'request_id': 'req-792dbee4-7008-44e8-b82e-4d57a3852d3c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:42.157292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ecfb2cfd5e4ed2a2c78e66a1390116', 'request_id': 'req-792dbee4-7008-44e8-b82e-4d57a3852d3c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:42.247930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] The token cache is being invalidate because project 02ecfb2cfd5e4ed2a2c78e66a1390116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:42.248465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:42.249395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-792dbee4-7008-44e8-b82e-4d57a3852d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:42.253493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 964/1922] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:41 2022] DELETE /identity/v3/projects/02ecfb2cfd5e4ed2a2c78e66a1390116 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:42.256430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.257804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.262962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.282534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:42.283739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:42.353301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 959/1923] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:42.361072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.361513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.361644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.381562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:42.382837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:42.391342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 965/1924] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:42.395895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.406843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78401a-9d1a-4282-8e0a-166626b58366 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.411461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.412123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.412406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.412805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.413455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:42.413763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.415351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.428668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 960/1925] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:42.432278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.441851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47055-0c3b-4c35-b015-34b71bf36107 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.456917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.457751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.458037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.458315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.459014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.461028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.564629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 966/1926] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:42 2022] POST /identity/v3/projects => generated 361 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:42.564730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.579308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea40baf-2d5d-4656-bbd3-7b63122f94fc admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.679770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.680442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.680751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.681027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.681726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.683687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.783356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 961/1927] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:42 2022] POST /identity/v3/users => generated 281 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:42.787500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.798754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae86e5-59cb-49e0-b0b9-200d42578a2c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.803488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.804136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.804415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.804720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.805439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.807022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.812227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:42.817555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 967/1928] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:42.821435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.831529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9af2-b395-40e4-b8da-e77377669375 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.836133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.836817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.837104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.837381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.838101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.839678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4096b25-ee78-47db-9669-4cf41c679c46 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:42.850159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 962/1929] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:42.854049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:42.865419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f18fb-dd01-47dd-a355-a62f2ab13f9b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:42.869931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:42.870826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:42.871106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:42.871382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] PATH_INFO: `/v3/projects/6db7a8845675411b9d731ac96ad133f4/users/0581ba2ec19d438db4fcf674bd9afe42/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:42.887609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db7a8845675411b9d731ac96ad133f4, user_id=0581ba2ec19d438db4fcf674bd9afe42, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:42.890334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:43.013824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 968/1930] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:42 2022] PUT /identity/v3/projects/6db7a8845675411b9d731ac96ad133f4/users/0581ba2ec19d438db4fcf674bd9afe42/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:43.024883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:43.025182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:43.025458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:43.052681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] MFA Rules not processed for user `0581ba2ec19d438db4fcf674bd9afe42`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:43.053966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:43.058104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:43.120122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 963/1931] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:43.154651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:43.156388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:43.226244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb400a0e-bc6d-44f1-9917-4ae57ea1f99d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:43.229947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:43.230585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:43.231401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:43.231679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:43.240302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:43.241779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5428d-d5a9-49d5-b8d7-59c9ac1b93bf service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:43.251180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 969/1932] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:43.265384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:43.327303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd6f98-f3f9-4ced-b006-3571ca195ebb None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:43.331906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:43.332827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:43.333098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:43.333381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] PATH_INFO: `/v3/users/0581ba2ec19d438db4fcf674bd9afe42` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:43.334609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0581ba2ec19d438db4fcf674bd9afe42)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:43.337105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:43.555621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0581ba2ec19d438db4fcf674bd9afe42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:43.563139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0581ba2ec19d438db4fcf674bd9afe42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:43.652598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0581ba2ec19d438db4fcf674bd9afe42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:43.653828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0581ba2ec19d438db4fcf674bd9afe42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:43.654433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0581ba2ec19d438db4fcf674bd9afe42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:43.671811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 964/1933] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:43 2022] DELETE /identity/v3/users/0581ba2ec19d438db4fcf674bd9afe42 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:43.676554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:43.677038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:43.677129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:43.694038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:43.695457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:43.769745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 970/1934] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:43.778554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:43.781030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:43.819130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:43.851546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ed2dd-bc99-4ce4-8dd3-5e95952fb7a9 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:43.855546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:43.856246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:43.856940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:43.857314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:43.868529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:43.870141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7030f2e-92f4-483a-a5f1-7cf2279c06ba service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:43.880810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 965/1935] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:44.334336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:44.335863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:44.359902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7a31b-a209-4478-a33d-344f309286f4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:44.367324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:44.368375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:44.368762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:44.369194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] PATH_INFO: `/v3/projects/6db7a8845675411b9d731ac96ad133f4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:44.370447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db7a8845675411b9d731ac96ad133f4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:44.372656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:44.500163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db7a8845675411b9d731ac96ad133f4', 'request_id': 'req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:44.505842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db7a8845675411b9d731ac96ad133f4', 'request_id': 'req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:44.512115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db7a8845675411b9d731ac96ad133f4', 'request_id': 'req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:44.618582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] The token cache is being invalidate because project 6db7a8845675411b9d731ac96ad133f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:44.618770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:44.619456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b96ab3b3-4f6f-4ca5-8f6d-80fde3391f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:44.621059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 971/1936] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:44 2022] DELETE /identity/v3/projects/6db7a8845675411b9d731ac96ad133f4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:44.626795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:44.627282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:44.627472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:44.645487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:44.646803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:44.708054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 966/1937] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:44.715623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:44.716103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:44.716253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:44.722687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:44.743424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:44.743424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:44.745127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 972/1938] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:44.750061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:44.760031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3b1db-b4c3-4a3a-8246-cd084364dca1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:44.764857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:44.765512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:44.765861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:44.766139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:44.767111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:44.767577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:44.769228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:44.782370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 967/1939] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:44.786905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:44.796902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc521a-8a1e-4dab-8011-660e2ff33621 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:44.801597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:44.802248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:44.802534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:44.802812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:44.803509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:44.805449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:44.963021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 973/1940] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:44 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:44.967423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:44.978076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13b732-0d40-47de-800b-fe92ac15fa62 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:45.046948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:45.047694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.047974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.048247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.048981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:45.050905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:45.253157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 968/1941] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:44 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:45.257157 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:45.268031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421686ee-be0a-4ce7-9f2f-e195fbc8ad52 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:45.272723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:45.273380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.273667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.273949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.274702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:45.276304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:45.286896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 974/1942] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:45.290885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:45.300903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e4658-9b58-40f0-b2ff-e7b261667b40 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:45.305464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:45.306110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.306395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.306670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.307389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:45.309008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3917835-de5c-47c5-aa00-d990c15395a7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:45.319029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 969/1943] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:45.323140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:45.333445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaac8e-e5ea-4c99-8829-a7e60fd1463c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:45.337983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:45.338856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.339137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.339410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] PATH_INFO: `/v3/projects/ba359eec1f4b479aac6e30e5ae0a7d9c/users/ef249eca516449ceb6ab1a3cdee510ab/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.354500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba359eec1f4b479aac6e30e5ae0a7d9c, user_id=ef249eca516449ceb6ab1a3cdee510ab, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:45.357186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:45.430432 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 975/1944] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:45 2022] PUT /identity/v3/projects/ba359eec1f4b479aac6e30e5ae0a7d9c/users/ef249eca516449ceb6ab1a3cdee510ab/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:45.440722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.441682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.441998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.476410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] MFA Rules not processed for user `ef249eca516449ceb6ab1a3cdee510ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:45.477717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:45.499432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:45.548016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 970/1945] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:45.560334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:45.562081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:45.621037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da547a-dff3-4961-a0fa-48edd553cc9d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:45.625381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:45.626031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:45.627901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:45.627901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:45.635419 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:45.636892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:45.646525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 976/1946] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:48.020349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:48.021948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:48.034672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8871c1-3261-4c64-a274-48f476a50ee0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:48.038412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:48.039172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:48.039368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:48.039626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:48.041461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:48.103589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:48.105108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef259106-d09c-4511-8efe-74f4cfea2302 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:48.114529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 971/1947] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:50.109077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:50.197552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c1791-dd71-4eca-92a7-f09f7145e44e service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:50.202931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:50.203851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:50.204133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:50.204435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] PATH_INFO: `/v3/users/ef249eca516449ceb6ab1a3cdee510ab` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:50.205763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef249eca516449ceb6ab1a3cdee510ab)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:50.207683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:50.447503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:50.451440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef249eca516449ceb6ab1a3cdee510ab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:50.458235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef249eca516449ceb6ab1a3cdee510ab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:50.459015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef249eca516449ceb6ab1a3cdee510ab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:50.561902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef249eca516449ceb6ab1a3cdee510ab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:50.562443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef249eca516449ceb6ab1a3cdee510ab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:50.580317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 977/1948] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:50 2022] DELETE /identity/v3/users/ef249eca516449ceb6ab1a3cdee510ab => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:50.584792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:50.585267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:50.585361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:50.603127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:50.604315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:50.665321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 972/1949] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:50.680466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:50.681915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:50.741579 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edc699-d3a5-4829-8bd8-7ac1eaea0440 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:50.745364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:50.746007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:50.746919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:50.747207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:50.756217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:50.757746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bfa9c-9e93-4d1a-a3e8-da1174db156e service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:50.767483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 978/1950] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:51.173302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:51.174807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:51.189600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378a829-2e85-4f6e-8be4-af87fdd893cc None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:51.194803 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:51.195947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.196365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.197387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] PATH_INFO: `/v3/projects/ba359eec1f4b479aac6e30e5ae0a7d9c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.199257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba359eec1f4b479aac6e30e5ae0a7d9c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:51.202161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:51.391890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba359eec1f4b479aac6e30e5ae0a7d9c', 'request_id': 'req-6b616429-f9f4-4fb8-b186-18bde441e401'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:51.403533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba359eec1f4b479aac6e30e5ae0a7d9c', 'request_id': 'req-6b616429-f9f4-4fb8-b186-18bde441e401'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:51.407956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:51.416778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba359eec1f4b479aac6e30e5ae0a7d9c', 'request_id': 'req-6b616429-f9f4-4fb8-b186-18bde441e401'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:51.521244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] The token cache is being invalidate because project ba359eec1f4b479aac6e30e5ae0a7d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:51.521538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:51.522162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-6b616429-f9f4-4fb8-b186-18bde441e401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:51.524092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 973/1951] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:51 2022] DELETE /identity/v3/projects/ba359eec1f4b479aac6e30e5ae0a7d9c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:51.529410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.529893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.529974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.548167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:51.550189 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:51.614647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 979/1952] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:51.623718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.624259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.624385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.646793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:51.648546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:51.658507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 974/1953] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:51.662706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:51.673484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cfbbc-3875-4594-8fcc-f673e450bc8f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:51.678188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:51.678846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.679137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.679412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.680139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:51.680446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:51.682111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:51.695778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 980/1954] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:29:51.699012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:51.708731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82362d1-7ec0-4781-9938-5a7068a79b4f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:51.713725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:51.714505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.714768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.715591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.716300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:51.718256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:51.847397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 975/1955] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:51 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:51.853787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:51.864177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120571-8ef7-4c90-99c4-74a41ebb9bd5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:51.917421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:51.927970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:51.928605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:51.928899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:51.929176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:51.929921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:51.931881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.072051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 981/1956] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:51 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:52.076209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:52.091759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2194-a1b2-4832-aa40-19a73c3601d0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:52.096794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:52.097492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.097769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.098047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.098786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:52.100414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.117514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:52.124439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 976/1957] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:52.128691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09e5d4-52f1-4acc-9af1-3d68132fc7b5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:52.133532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:52.134204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.134491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.134772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.135514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:52.137212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42043a6-30cd-42ec-b886-a392ce8b4225 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.155490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 982/1958] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:52.163433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:52.178477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f788e-33a9-4071-ae51-93fdd9d745b5 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:52.185167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:52.186065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.186354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.186635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] PATH_INFO: `/v3/projects/44089f44ea384df88ea8e9411dc94476/users/7e44a60fc7244d6d88a20aeac53c3353/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.203290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=44089f44ea384df88ea8e9411dc94476, user_id=7e44a60fc7244d6d88a20aeac53c3353, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:52.206027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.305890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 977/1959] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:52 2022] PUT /identity/v3/projects/44089f44ea384df88ea8e9411dc94476/users/7e44a60fc7244d6d88a20aeac53c3353/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:52.316632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.316942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.317219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.352504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] MFA Rules not processed for user `7e44a60fc7244d6d88a20aeac53c3353`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:52.353957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:52.354110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:52.355951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:52.432771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 983/1960] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:52.443071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0520a2-b976-427a-8b15-6f19f0c127ee admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:52.444278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:52.445780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:52.448395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:52.449427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.450172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.450533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] PATH_INFO: `/v3/users/e9f5e91a02354cd8ad6c3133682db17a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.460726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f5e91a02354cd8ad6c3133682db17a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:52.462680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.529210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b937f0-94d0-4535-847d-a7290756db4a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:52.532943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:52.533625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:52.533871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:52.534147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:52.545248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:52.546735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:52.558083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 984/1961] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:52.755839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f5e91a02354cd8ad6c3133682db17a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:52.764930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f5e91a02354cd8ad6c3133682db17a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.028357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:53.029788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:53.043292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400dd21-d634-4aba-ac82-db8739c3f0d9 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:53.047775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:53.048659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:53.048942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:53.049213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] PATH_INFO: `/v3/users/d1bac82c0396416aa875baffb06d3268` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:53.050385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bac82c0396416aa875baffb06d3268)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:53.052276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:53.059767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f5e91a02354cd8ad6c3133682db17a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.060403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f5e91a02354cd8ad6c3133682db17a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.061212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ac9ab21f-5847-4181-b180-354ecfa7315c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f5e91a02354cd8ad6c3133682db17a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.085558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:53.089974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 978/1962] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:52 2022] DELETE /identity/v3/users/e9f5e91a02354cd8ad6c3133682db17a => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:53.094515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:53.094794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:53.095068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:53.112422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:53.113647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:53.192639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 979/1963] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:53.207744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:53.210667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:53.275411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293bd4f-f08f-43d5-b758-250a4a7f2a1a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:53.279140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:53.279800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:53.280084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:53.280371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:53.298184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:53.299827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d35a-43f7-4935-a59f-23a604cc0878 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:53.317745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 980/1964] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:53.319071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bac82c0396416aa875baffb06d3268'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.326864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bac82c0396416aa875baffb06d3268'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.327693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bac82c0396416aa875baffb06d3268'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.329538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:53.402965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bac82c0396416aa875baffb06d3268'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.405752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bac82c0396416aa875baffb06d3268'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:53.427112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 985/1965] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:53 2022] DELETE /identity/v3/users/d1bac82c0396416aa875baffb06d3268 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:53.434495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:53.434495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:53.434495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:53.452298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:53.453592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:53.517103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 981/1966] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:53.526765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:53.528350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:53.604780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9d50-f44f-477f-ba27-619d0f41a841 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:53.604780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:53.604780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:53.604780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:53.606176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:53.616565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:53.618225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:53.628921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 986/1967] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:29:54.215595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:54.216621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:54.218557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:54.220031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:54.242666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22bac0-9c8d-4ae8-9a42-dc53e2d65310 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:54.248868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:54.249767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:54.250060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:54.250334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] PATH_INFO: `/v3/projects/f009a7a071f146bf8115da4fcf328e2a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:54.251844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f009a7a071f146bf8115da4fcf328e2a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:54.252735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb848b-dba8-468a-8ae0-c8b0ddfdd539 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:54.253912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:54.270140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:54.270140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:54.270140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:54.270140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] PATH_INFO: `/v3/projects/0ea4e2754da54585a3796e6ba91d4237` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:54.271609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea4e2754da54585a3796e6ba91d4237)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:54.273795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:54.367595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:54.375000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f009a7a071f146bf8115da4fcf328e2a', 'request_id': 'req-89acf211-0ae8-4346-9162-70912ab18c5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.381350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f009a7a071f146bf8115da4fcf328e2a', 'request_id': 'req-89acf211-0ae8-4346-9162-70912ab18c5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.391569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f009a7a071f146bf8115da4fcf328e2a', 'request_id': 'req-89acf211-0ae8-4346-9162-70912ab18c5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.407694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea4e2754da54585a3796e6ba91d4237', 'request_id': 'req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.413346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea4e2754da54585a3796e6ba91d4237', 'request_id': 'req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.419726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea4e2754da54585a3796e6ba91d4237', 'request_id': 'req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.509657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] The token cache is being invalidate because project f009a7a071f146bf8115da4fcf328e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:54.509951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.510608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.512613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 982/1968] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:54 2022] DELETE /identity/v3/projects/f009a7a071f146bf8115da4fcf328e2a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:54.516196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:54.517604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:54.567143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] The token cache is being invalidate because project 0ea4e2754da54585a3796e6ba91d4237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:54.567433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.568182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.570058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 987/1969] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:54 2022] DELETE /identity/v3/projects/0ea4e2754da54585a3796e6ba91d4237 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:54.573967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:54.575393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:54.595095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acf211-0ae8-4346-9162-70912ab18c5f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:54.654548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3083a0-f9f5-4f10-886e-2bf4dc399bf7 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:54.659800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:54.660843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:54.661154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:54.661551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] PATH_INFO: `/v3/users/4b281101a58d4ff99242d70f27b3e572` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:54.661960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:54.662889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b281101a58d4ff99242d70f27b3e572)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:54.663021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:54.663336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:54.663660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] PATH_INFO: `/v3/users/32238d194cab485089e45f38b2780746` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:54.664860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:54.666286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=32238d194cab485089e45f38b2780746)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:54.667588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:54.670447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:54.956000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32238d194cab485089e45f38b2780746'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.956418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b281101a58d4ff99242d70f27b3e572'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.966627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32238d194cab485089e45f38b2780746'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.967068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b281101a58d4ff99242d70f27b3e572'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:54.967407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32238d194cab485089e45f38b2780746'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.039935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32238d194cab485089e45f38b2780746'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.040564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32238d194cab485089e45f38b2780746'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.060853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 988/1970] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:54 2022] DELETE /identity/v3/users/32238d194cab485089e45f38b2780746 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:55.159256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b281101a58d4ff99242d70f27b3e572'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.159899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b281101a58d4ff99242d70f27b3e572'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.160617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-32df8328-ae68-41ce-921d-b0ba749bea73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b281101a58d4ff99242d70f27b3e572'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.180132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 983/1971] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:54 2022] DELETE /identity/v3/users/4b281101a58d4ff99242d70f27b3e572 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:55.183042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:55.658644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:55.664735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:55.746888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9c3c-afb7-4abf-9731-1fe5950d32ca admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:55.759066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:55.759600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:55.759600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:55.801375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] MFA Rules not processed for user `7e44a60fc7244d6d88a20aeac53c3353`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:55.803584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:55.816354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:55.817503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:55.817864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:55.818214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] PATH_INFO: `/v3/projects/65cb083306cd43e2b894f478cd1582eb` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:55.819859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cb083306cd43e2b894f478cd1582eb)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:55.823171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:55.886612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 984/1972] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:55.891529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:55.891529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:55.905802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2de2c-d730-49bb-9efd-14160ba967bd None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:55.911196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:55.911396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:55.911563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:55.911917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] PATH_INFO: `/v3/projects/a4923b5e049c468caa5d9017debacdc4` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:55.913213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4923b5e049c468caa5d9017debacdc4)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:55.915166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:55.964667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:55.985671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cb083306cd43e2b894f478cd1582eb', 'request_id': 'req-bbb1e428-846e-4019-9451-54330b99b4cc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.992607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cb083306cd43e2b894f478cd1582eb', 'request_id': 'req-bbb1e428-846e-4019-9451-54330b99b4cc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:55.999725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cb083306cd43e2b894f478cd1582eb', 'request_id': 'req-bbb1e428-846e-4019-9451-54330b99b4cc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.084330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4923b5e049c468caa5d9017debacdc4', 'request_id': 'req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.095952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4923b5e049c468caa5d9017debacdc4', 'request_id': 'req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.102814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4923b5e049c468caa5d9017debacdc4', 'request_id': 'req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.103082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] The token cache is being invalidate because project 65cb083306cd43e2b894f478cd1582eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:56.103382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.104031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.108787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 989/1973] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:55 2022] DELETE /identity/v3/projects/65cb083306cd43e2b894f478cd1582eb => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:56.109819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.111172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:56.173613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1e428-846e-4019-9451-54330b99b4cc admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:56.231799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:56.232720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.233003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.233530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] PATH_INFO: `/v3/users/167d9a7415484fc1aae152c891c96764` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.234970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] RBAC: Authorizing `identity:delete_user(user_id=167d9a7415484fc1aae152c891c96764)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:56.237179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:56.241720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] The token cache is being invalidate because project a4923b5e049c468caa5d9017debacdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:56.244788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.244788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9698d3d8-f682-4dda-a1c7-6dbc6492d04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.244788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 985/1974] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:55 2022] DELETE /identity/v3/projects/a4923b5e049c468caa5d9017debacdc4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:56.249288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35140-abd1-4c57-af56-3f0d7abf315a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.249573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35140-abd1-4c57-af56-3f0d7abf315a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.249848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35140-abd1-4c57-af56-3f0d7abf315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.268551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c1f35140-abd1-4c57-af56-3f0d7abf315a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:56.269851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c1f35140-abd1-4c57-af56-3f0d7abf315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:56.286606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 986/1975] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:56.294888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.295179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.295462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.314397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:56.315815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:56.324166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 987/1976] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:56.328299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.337584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4315763-26f6-4cbd-a6d0-bc8cae4daf96 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:56.342862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:56.343512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.349004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.349004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.349004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:56.349004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:56.349004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:56.360411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 988/1977] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:29:56.364673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.375284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3c82b-0e18-47c9-a975-f733f2f28204 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:56.380612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:56.383913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.383913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.383913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.383913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:56.384647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:56.445856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167d9a7415484fc1aae152c891c96764'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.454422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167d9a7415484fc1aae152c891c96764'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.455171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167d9a7415484fc1aae152c891c96764'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.557509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 989/1978] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:56 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:56.562708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.582776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167d9a7415484fc1aae152c891c96764'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.583961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167d9a7415484fc1aae152c891c96764'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:56.586355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:56.624023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 990/1979] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:29:56 2022] DELETE /identity/v3/users/167d9a7415484fc1aae152c891c96764 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:56.635629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:56.681514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc8e0-c6a0-482f-97b8-203fcaa48cbc admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:56.766871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:56.767598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.768008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.768375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.769551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:56.771775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:56.940512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 990/1980] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:56 2022] POST /identity/v3/users => generated 283 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:56.944991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.946689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:56.960774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a8e44-24f9-4580-9fd2-050531bd980f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:56.966298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:56.970557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:56.970557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:56.970557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:56.970557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:56.970557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:56.980282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 991/1981] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:56.985281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:56.996544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594a1d4-b03b-4c3b-81df-810b784f645f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.011545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.012503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.012925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.018991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.020369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.022690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274434e-4d3c-4466-9503-d105dfa37791 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.039664 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 991/1982] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:57.040936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.053493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0da18-e0bd-4062-9c15-5f0f174f4150 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.059017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.059935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.065459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.065459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] PATH_INFO: `/v3/projects/18af4dbd7f224c08a9adb41839babb82/users/7d129584f9344d3fa59603021df033a7/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.077257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18af4dbd7f224c08a9adb41839babb82, user_id=7d129584f9344d3fa59603021df033a7, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.080132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.158603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 992/1983] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:57 2022] PUT /identity/v3/projects/18af4dbd7f224c08a9adb41839babb82/users/7d129584f9344d3fa59603021df033a7/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:57.169909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.170203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.170481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.200442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] MFA Rules not processed for user `7d129584f9344d3fa59603021df033a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:57.201728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:57.252522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.254275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:57.275858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 992/1984] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:57.285842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.338714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127aae-70ed-4028-bae0-cbc1674f4928 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.344282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.345397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.345670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.345950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] PATH_INFO: `/v3/projects/b6048dcd33a3438a82a7cc93a564160b` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.347136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6048dcd33a3438a82a7cc93a564160b)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.349101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.354404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67ffbc-1b61-4374-8862-873e9b078216 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.359167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.359823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.360102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.360375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.361091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.362961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.475790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6048dcd33a3438a82a7cc93a564160b', 'request_id': 'req-7ac32f38-1103-44db-8547-2afd29f49570'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:57.481764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6048dcd33a3438a82a7cc93a564160b', 'request_id': 'req-7ac32f38-1103-44db-8547-2afd29f49570'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:57.488324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6048dcd33a3438a82a7cc93a564160b', 'request_id': 'req-7ac32f38-1103-44db-8547-2afd29f49570'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:57.532836 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 993/1985] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/projects => generated 367 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:57.533942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.614721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309352-cc39-42ad-b2bc-29c55b6cb887 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.626392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.627252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.627719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.628182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.629167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.631489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.635215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] The token cache is being invalidate because project b6048dcd33a3438a82a7cc93a564160b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:29:57.641784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:57.647244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7ac32f38-1103-44db-8547-2afd29f49570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:29:57.657403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 993/1986] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:29:57 2022] DELETE /identity/v3/projects/b6048dcd33a3438a82a7cc93a564160b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:57.659747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95db5-e870-4578-b19c-64db6a7b34de None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.660180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95db5-e870-4578-b19c-64db6a7b34de None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.660607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95db5-e870-4578-b19c-64db6a7b34de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.687332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cad95db5-e870-4578-b19c-64db6a7b34de None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:57.689116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cad95db5-e870-4578-b19c-64db6a7b34de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:57.715822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 994/1987] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:57.718397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.718697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.719174 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.754155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 994/1988] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:57.754611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:57.756109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:57.756272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.758224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:57.774584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 995/1989] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:29:57.775000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc45d5e-3113-4500-992c-5276edcc525d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.785149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.788125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caa712-d23f-4af4-a78f-80a95e0e44ab None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.791101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.792539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.792948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.793298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.793644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.794264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.794404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.794839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.795186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.795997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:29:57.796321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.796440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.798158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.812974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 996/1990] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:29:57.817729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.843410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 995/1991] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:57.844879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:57.854806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d98cd-1e00-42a9-baff-217bede5a7c1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.855102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.855771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.856052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.856324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.857053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.859362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.870339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:57.874622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3ac8-58a4-4580-99a5-9667090eeaaa admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:57.882692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:57.883728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:57.884089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:57.884427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] PATH_INFO: `/v3/projects/dbcd503f9d2d494b8cb28a678b987d27/users/cd09b4d862164625b0ea62a2ff904769/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:57.907243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcd503f9d2d494b8cb28a678b987d27, user_id=cd09b4d862164625b0ea62a2ff904769, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:57.910807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:57.960424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:57.970862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 997/1992] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:57.972216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.040342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d635-f7fe-4c13-9aab-1bd7650946b9 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.043457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 996/1993] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:57 2022] PUT /identity/v3/projects/dbcd503f9d2d494b8cb28a678b987d27/users/cd09b4d862164625b0ea62a2ff904769/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:58.045889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.121025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.122299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6486b-2dd3-4e25-8f87-c7c35e8729d3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.122441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.122607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.122937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.123732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.126235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.132084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.132084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.132084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.132084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.132400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.132400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.143577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 997/1994] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:29:58.147450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.157700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402cbc-5f00-4921-ab45-48d92e245ded admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.162323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.163190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.163473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.163750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] PATH_INFO: `/v3/projects/dbcd503f9d2d494b8cb28a678b987d27/users/cd09b4d862164625b0ea62a2ff904769/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.165791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcd503f9d2d494b8cb28a678b987d27, user_id=cd09b4d862164625b0ea62a2ff904769, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.169003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e5d39-e317-4a70-ac2a-f59e5097f43d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.190047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 998/1995] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:58 2022] PUT /identity/v3/projects/dbcd503f9d2d494b8cb28a678b987d27/users/cd09b4d862164625b0ea62a2ff904769/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:29:58.194189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.194864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.195303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.222695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] MFA Rules not processed for user `cd09b4d862164625b0ea62a2ff904769`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:58.223949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:58.315729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 998/1996] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:57 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:29:58.316602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.329327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 999/1997] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:58.333605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467608dd-a3de-4b76-aa1f-5a35873bbaf6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.339402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.341548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:58.430994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.431224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c24c-d71c-45fd-92d0-fe6a5947db74 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.432282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.433248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.434477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.435411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.436703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.437334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.437508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.439683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.440037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.453077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 999/1998] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:58.456354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.457338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.458859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.478612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b141-7acf-4869-88bc-a4fade105557 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.482065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1000/1999] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:58.485392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.486063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.486344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.486614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.487407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.489033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7101435-7cf8-482d-8415-0459f15cdf3d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.500515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1000/2000] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:58.504800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.516256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a12c-cd2c-40f1-970a-763be6d86f82 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.521608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.522550 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.522833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.523136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] PATH_INFO: `/v3/projects/dbb61f73ff774ec1b2a47cc32beeebec/users/0ec158e60b2d41ae854536f88f03a9a3/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.538413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb61f73ff774ec1b2a47cc32beeebec, user_id=0ec158e60b2d41ae854536f88f03a9a3, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.541099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.630544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1001/2001] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:58 2022] PUT /identity/v3/projects/dbb61f73ff774ec1b2a47cc32beeebec/users/0ec158e60b2d41ae854536f88f03a9a3/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:58.643479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.643740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.644307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.671818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] MFA Rules not processed for user `0ec158e60b2d41ae854536f88f03a9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:58.673176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:58.741928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.745390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1001/2002] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:58.749468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.819977 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72db5-1bf9-4a7a-b8ce-98f1ae949123 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.824236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7e349-d8a7-46c4-b187-6e89e3423f2c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:58.828912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.833795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:58.833795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.834020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:58.834119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.834119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.834301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:58.834301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:58.834466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.834543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:58.834613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.903232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:58.904861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:58.913992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1002/2003] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:29:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:29:58.941831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1002/2004] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:58 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:58.946170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:58.967044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d43838-3ece-4d96-9746-0e025d166a05 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.047161 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.047824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.048097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.048367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.049102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.051334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.082861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:59.165211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1003/2005] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:58 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:59.166412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.177628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdab-73d8-4d83-a0bf-9203c181b764 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.182387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.183110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.183469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.183841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.184749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.186688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.200298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1003/2006] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:59.201567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.212879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadd99-1dd1-4e05-8820-105398a83148 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.219869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.220529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.220837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.221119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.221847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.223448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b8186-3607-4ac6-88c7-8cfdd1a4a1a7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.234987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1004/2007] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:59.239120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.249480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156bac-8276-4f27-96a2-f946634854e3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.254430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.255308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.255590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.255868 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] PATH_INFO: `/v3/projects/aaeae6872bcc429399fc34209f233395/users/680529e3bbcf46d4a6cfcb83d1e8616c/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.271167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaeae6872bcc429399fc34209f233395, user_id=680529e3bbcf46d4a6cfcb83d1e8616c, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.273864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.384916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1004/2008] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:59 2022] PUT /identity/v3/projects/aaeae6872bcc429399fc34209f233395/users/680529e3bbcf46d4a6cfcb83d1e8616c/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:29:59.404626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.405290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.405572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.407977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:59.434238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] MFA Rules not processed for user `680529e3bbcf46d4a6cfcb83d1e8616c`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:29:59.435795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:29:59.503981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1005/2009] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:29:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:29:59.508315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.595854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b406-ec4a-46bd-bd5c-4610dec84c62 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.602575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.603307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.603644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.603933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.604682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.606714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.720237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1005/2010] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:29:59 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:59.723392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.742139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702de422-4e5f-41cc-93b4-2446185b860e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.799879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.800642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.800922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.801204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.801923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.803850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.920183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1006/2011] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:29:59 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:29:59.921462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.934820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d2028-8429-4998-90fa-c6fd745bf563 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.941041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.941841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.942137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.942408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.943151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:29:59.945279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:29:59.958162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1006/2012] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:29:59.962392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:29:59.963818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:29:59.975034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb0bb-3695-44f3-9a74-3029cb6877ff admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:29:59.980146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:29:59.981833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:29:59.982196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:29:59.982541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] PATH_INFO: `/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8/users/8ec4486473264adcade99cab42893710/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:29:59.999531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6e1ceab1c40ab9cf528a8f1a16cd8, user_id=8ec4486473264adcade99cab42893710, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:00.002504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:00.096766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:00.102716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1007/2013] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:29:59 2022] PUT /identity/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8/users/8ec4486473264adcade99cab42893710/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:00.107175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:00.117529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e923e8-eb1b-4352-a996-f35494cda562 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:00.176845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:00.177516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:00.177797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:00.178072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:00.178824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:00.180427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004003a-ed4c-4bd3-8400-2f5a08e79fde admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:00.194694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1007/2014] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:00.197400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:00.207341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c29a5-d340-4c2c-a10b-c4d193f06a91 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:00.212101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:00.213118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:00.213342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:00.213621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] PATH_INFO: `/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8/users/8ec4486473264adcade99cab42893710/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:00.215675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6e1ceab1c40ab9cf528a8f1a16cd8, user_id=8ec4486473264adcade99cab42893710, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:00.218359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:00.229462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1008/2015] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:00 2022] PUT /identity/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8/users/8ec4486473264adcade99cab42893710/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:00.240339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:00.240773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:00.241055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:00.268735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] MFA Rules not processed for user `8ec4486473264adcade99cab42893710`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:00.269934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:00.342357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1008/2016] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:00.354660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:00.410608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac738e-6d1c-4353-b252-a38fdafde294 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:00.414633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:00.415281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:00.415562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:00.416473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:00.479032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:00.480741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:00.490576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1009/2017] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:00.531222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:03.720272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:03.723026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:03.740335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6150c4-c1d7-4d55-908c-2fe4c44deb01 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:03.746839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:03.747957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:03.748395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:03.749018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:03.799608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:03.812818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:03.814486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:03.825358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1009/2018] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:05.644493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:05.653649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ed631-3baf-4e83-abcc-a23b2169ad66 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:05.657407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:05.658055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:05.658349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:05.658609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:05.669071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:05.670495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e7fc-4729-43a0-bfd9-08bee06b1ca0 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:05.685001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1010/2019] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:07.275372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:07.353193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a94939-eb6d-4ab6-953d-981c8b25f4f1 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:07.359346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:07.360229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:07.360518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:07.360837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] PATH_INFO: `/v3/users/7d129584f9344d3fa59603021df033a7` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:07.362051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d129584f9344d3fa59603021df033a7)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:07.363995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:07.625331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d129584f9344d3fa59603021df033a7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:07.632685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d129584f9344d3fa59603021df033a7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:07.633407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d129584f9344d3fa59603021df033a7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:07.721578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d129584f9344d3fa59603021df033a7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:07.722463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d129584f9344d3fa59603021df033a7'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:07.758306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1010/2020] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:07 2022] DELETE /identity/v3/users/7d129584f9344d3fa59603021df033a7 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:07.768032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6108f9-205f-4aa8-bbd7-a096c54b513a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:07.768555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6108f9-205f-4aa8-bbd7-a096c54b513a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:07.768784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6108f9-205f-4aa8-bbd7-a096c54b513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:07.786763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9c6108f9-205f-4aa8-bbd7-a096c54b513a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:07.787985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9c6108f9-205f-4aa8-bbd7-a096c54b513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:07.818224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:07.859358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1011/2021] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:07.869037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:07.870470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:07.928940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8eecf-b5dc-4310-a0da-86bead18a05c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:07.933416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:07.941645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:07.941645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:07.941645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:07.944690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:07.946207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2135e-73d1-49d0-b6ba-cfbb27d0f919 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:07.957887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1011/2022] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:07.989455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:07.989643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:07.989849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:07.991443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1012/2023] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:30:08.002519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.015931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.015931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.026085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:08.036406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:08.038382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.115496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1012/2024] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:30:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:08.127502 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:08.129423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.148179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:08.150847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.162233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b4ae7-fb83-459e-9103-4ff6549fb69c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:08.168178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:08.168297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcecfb3-7c02-44ef-ac2e-9ebe6ab1dbed None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:08.169029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.169522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.169996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.173306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:08.182413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.182413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.182413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] PATH_INFO: `/v3/users/9726e11bda974054a1e972c097b1ea1d` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.182413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9726e11bda974054a1e972c097b1ea1d)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:08.182413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:08.182792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:08.182792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:08.204481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1013/2025] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:08.438153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9726e11bda974054a1e972c097b1ea1d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.445183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9726e11bda974054a1e972c097b1ea1d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.445854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9726e11bda974054a1e972c097b1ea1d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.518251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:08.519742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.535168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135d200-0359-46cb-be0f-fa1cdda36ac4 service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:08.539797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:08.546675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.546675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.546675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] PATH_INFO: `/v3/projects/18af4dbd7f224c08a9adb41839babb82` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.546675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18af4dbd7f224c08a9adb41839babb82)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:08.546675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:08.565457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9726e11bda974054a1e972c097b1ea1d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.565848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9726e11bda974054a1e972c097b1ea1d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.584686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1013/2026] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:08 2022] DELETE /identity/v3/users/9726e11bda974054a1e972c097b1ea1d => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:08.673043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18af4dbd7f224c08a9adb41839babb82', 'request_id': 'req-f555a74b-c264-4ffa-aad9-c2f55745bc13'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.679445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18af4dbd7f224c08a9adb41839babb82', 'request_id': 'req-f555a74b-c264-4ffa-aad9-c2f55745bc13'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.685391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18af4dbd7f224c08a9adb41839babb82', 'request_id': 'req-f555a74b-c264-4ffa-aad9-c2f55745bc13'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.764801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] The token cache is being invalidate because project 18af4dbd7f224c08a9adb41839babb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:08.765088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.765782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:08.767347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1014/2027] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:08 2022] DELETE /identity/v3/projects/18af4dbd7f224c08a9adb41839babb82 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:08.771630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:08.773088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.837741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad6acc-fddd-4925-9069-3d5eb1649dc0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:08.878119 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:08.879814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:08.898037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:08.898990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.899205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.899483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] PATH_INFO: `/v3/users/cd09b4d862164625b0ea62a2ff904769` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.900871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd09b4d862164625b0ea62a2ff904769)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:08.903042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:08.951097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555a74b-c264-4ffa-aad9-c2f55745bc13 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:08.955849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:08.956749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:08.957031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:08.957303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] PATH_INFO: `/v3/projects/986f0a9e430e4caaaed4affc2d4b8aeb` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:08.958485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986f0a9e430e4caaaed4affc2d4b8aeb)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:08.960682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.064832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:09.221853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:09.225648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd09b4d862164625b0ea62a2ff904769'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.226792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986f0a9e430e4caaaed4affc2d4b8aeb', 'request_id': 'req-79094ed5-7cfa-4490-916f-85a79c9d0ca3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.233621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986f0a9e430e4caaaed4affc2d4b8aeb', 'request_id': 'req-79094ed5-7cfa-4490-916f-85a79c9d0ca3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.239514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd09b4d862164625b0ea62a2ff904769'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.249778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986f0a9e430e4caaaed4affc2d4b8aeb', 'request_id': 'req-79094ed5-7cfa-4490-916f-85a79c9d0ca3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.249911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd09b4d862164625b0ea62a2ff904769'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.344354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] The token cache is being invalidate because project 986f0a9e430e4caaaed4affc2d4b8aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:09.344672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.345301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-79094ed5-7cfa-4490-916f-85a79c9d0ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.346970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1015/2028] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:08 2022] DELETE /identity/v3/projects/986f0a9e430e4caaaed4affc2d4b8aeb => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:09.357613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.358189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.358463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.378251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:09.379464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:09.401714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd09b4d862164625b0ea62a2ff904769'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.402695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-68b81cfb-40c0-4e28-ab7b-e11909ef3a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd09b4d862164625b0ea62a2ff904769'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:09.432186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1014/2029] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:08 2022] DELETE /identity/v3/users/cd09b4d862164625b0ea62a2ff904769 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:09.460739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1016/2030] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:09.463058 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.463525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.465173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.486252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:09.487588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:09.555252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1015/2031] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:09.559841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.570102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae014df-ba03-41d8-b9ef-04fb0105a56e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.574891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.575548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.575832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.576127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.576916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:09.577225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.578858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.592229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1017/2032] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:30:09.610462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.619895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a039d5-2f21-48fc-ab06-91b806418d1e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.624688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.625342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.625625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.625898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.626596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.628547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.721278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1016/2033] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:09 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:09.722421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.733611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46f8e-8ee6-42b0-b31d-9f785027e8ff admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.833016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.833881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.834824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.837824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.837824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.837824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.898493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.900461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:09.924107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d33bfb-283b-4b0a-97d2-c3bc47b7fdaf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.926444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1018/2034] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:09 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:09.930314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.936269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.937530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.937916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.938370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] PATH_INFO: `/v3/projects/dbcd503f9d2d494b8cb28a678b987d27` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.940565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbcd503f9d2d494b8cb28a678b987d27)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.941554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e157-2f81-4159-9872-a1bc19d0ab29 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.943197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.947448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.948113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.948396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.949329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.949981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.951791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:09.965190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1019/2035] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:09.967879 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:09.978567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040458a4-2adb-477f-a2a1-01a42adcb6c3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:09.983665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:09.984324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:09.984631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:09.984912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:09.985636 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:09.987219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.004627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1020/2036] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:10.008052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.022251 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aa730-aace-422c-86c6-f642e6af23a5 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.028991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.029907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.030187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.030461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] PATH_INFO: `/v3/projects/1ae23d2cb99648a89af0efc876c1410e/users/0088ea2a4d3b45b19fd80df3a65fecb3/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.046809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae23d2cb99648a89af0efc876c1410e, user_id=0088ea2a4d3b45b19fd80df3a65fecb3, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.051587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3416ca-058c-4fc9-a4cd-871a5f4bb4e5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.203108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1021/2037] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:10 2022] PUT /identity/v3/projects/1ae23d2cb99648a89af0efc876c1410e/users/0088ea2a4d3b45b19fd80df3a65fecb3/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:10.210838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbcd503f9d2d494b8cb28a678b987d27', 'request_id': 'req-f9eef722-f4ee-4ab6-8471-24438cfe25a9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:10.212990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.213287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.213572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.217031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbcd503f9d2d494b8cb28a678b987d27', 'request_id': 'req-f9eef722-f4ee-4ab6-8471-24438cfe25a9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:10.223642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbcd503f9d2d494b8cb28a678b987d27', 'request_id': 'req-f9eef722-f4ee-4ab6-8471-24438cfe25a9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:10.241792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] MFA Rules not processed for user `0088ea2a4d3b45b19fd80df3a65fecb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:10.243013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:10.304472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] The token cache is being invalidate because project dbcd503f9d2d494b8cb28a678b987d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:10.304472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:10.305271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f9eef722-f4ee-4ab6-8471-24438cfe25a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:10.310510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1017/2038] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:09 2022] DELETE /identity/v3/projects/dbcd503f9d2d494b8cb28a678b987d27 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:10.312159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0775-4373-4a59-b8ce-760ffcc3a921 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.312453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0775-4373-4a59-b8ce-760ffcc3a921 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.312770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0775-4373-4a59-b8ce-760ffcc3a921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.313895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1022/2039] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:10.317901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.331665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ffdc0775-4373-4a59-b8ce-760ffcc3a921 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:10.333692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ffdc0775-4373-4a59-b8ce-760ffcc3a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:10.380657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529b987-14c4-46e8-8c90-db1afc46e3c3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.385363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.386023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.386309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.387204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.387890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.389861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.395803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1018/2040] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:10.403305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.403600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.404197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.422386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:10.423591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:10.431815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1019/2041] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:10.436158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.445182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730a245-2483-442b-b0f3-c0ffdc94bb85 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.450646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.451302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.451582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.451854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.452575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:10.452913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.454534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.466287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1020/2042] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:30:10.470556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.480074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac92aa1-6654-43eb-bb9e-c91b50914cfb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.485349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.485993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.486279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.486556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.487469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.489471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.504181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1023/2043] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:10.511871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.513547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:10.528244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:10.617151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73884cd6-e18c-4e51-90b3-f1894bbe9e8c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.625782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1021/2044] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:10.625900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.627678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.627969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.628250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.628977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.629095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.631732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.651111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:10.733882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850f5f1-dcaa-4812-bdd8-a804dfdb08f8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.740362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.741136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.741516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.741677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.742373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.744302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.777713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1024/2045] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/users => generated 298 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:10.781051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.819786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1022/2046] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:10 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:10.822388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c526ec-a866-4cce-904b-e3e66085dd9c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.824574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.829698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.830308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.830799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.831210 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.832395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.833676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.848025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1025/2047] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:10.848259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21ab5-2bab-4365-aa1e-2374fb1973c4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.851895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.853827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.854498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.854776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.855084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.855826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.857461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.863524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c135d2-9944-4aa0-86b7-5e4f84abcd14 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.868374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.872994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1023/2048] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:10.873103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.873103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.873103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] PATH_INFO: `/v3/projects/2e1172b4d5244754afb2021a177ada9c/users/26b6392796bf4f39af67dda942604fd7/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.875744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.886895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1172b4d5244754afb2021a177ada9c, user_id=26b6392796bf4f39af67dda942604fd7, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.887118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e656-49ef-4f45-9823-201b8a4645a7 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.889779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.892022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.892737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.893032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.893307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.894032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.895615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.907464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1024/2049] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:10.911883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:10.923532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca1d1-684b-44a9-b950-a8971bef9501 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:10.928318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:10.929225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:10.930718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:10.930718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] PATH_INFO: `/v3/projects/bb0c7b47d3704f5e9b05d45e3595281b/users/e2101751451042b7b9c4c3df826e496e/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:10.945283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0c7b47d3704f5e9b05d45e3595281b, user_id=e2101751451042b7b9c4c3df826e496e, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:10.948246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef522-4ce6-440b-902c-9fc43d36892e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:10.973009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1026/2050] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:10 2022] PUT /identity/v3/projects/2e1172b4d5244754afb2021a177ada9c/users/26b6392796bf4f39af67dda942604fd7/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:10.976780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.038518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abb-1b40-4e88-a13b-2f7000a49eab admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.043349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.044057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.044342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.044676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.045433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.047016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.056726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1027/2051] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:11.061163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.070548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6139115-9749-46f9-8fc0-5b51de7400ca admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.075477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.076373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.076694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.076972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] PATH_INFO: `/v3/projects/2e1172b4d5244754afb2021a177ada9c/users/26b6392796bf4f39af67dda942604fd7/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.078988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1172b4d5244754afb2021a177ada9c, user_id=26b6392796bf4f39af67dda942604fd7, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.084733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1025/2052] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:10 2022] PUT /identity/v3/projects/bb0c7b47d3704f5e9b05d45e3595281b/users/e2101751451042b7b9c4c3df826e496e/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:11.084870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cc825-485e-4009-8664-0ef151cd6bb0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.094718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.095137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1028/2053] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:11 2022] PUT /identity/v3/projects/2e1172b4d5244754afb2021a177ada9c/users/26b6392796bf4f39af67dda942604fd7/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:11.095255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.095346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.105769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.106062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.106338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.124539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] MFA Rules not processed for user `e2101751451042b7b9c4c3df826e496e`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:11.125839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:11.134003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] MFA Rules not processed for user `26b6392796bf4f39af67dda942604fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:11.137117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:11.198148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1026/2054] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:11.202582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.211587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1029/2055] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:11.222717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.224073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:11.264355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f9078-58c6-4920-98ba-95b3aaa383de None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.269158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.269748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.270034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.270304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.270983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.272903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.299528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96377d42-a7d2-4e8e-90c1-0c2de76903e0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.303224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.303876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.304146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.304420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.313878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.315302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.325258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1030/2056] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:11.425841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.431807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1027/2057] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:11 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:11.435284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5dee4-7940-4639-bae8-a1af5200219e service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.441737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.484800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.485492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.485798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.486074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.487975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:11.503725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fad6a2-7d53-410a-baaf-23f54242a78e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.508560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.509254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.509540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.509817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.510512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.512504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.546764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.548221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.557089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1031/2058] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:11.620268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1028/2059] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:11 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:11.629326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.640687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f7b1-6872-4cc2-82fc-434d6def89fe service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.645720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.646094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.646378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.646653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.647384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.649009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.660465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1032/2060] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:11.664945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.676844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a3c0-17ea-4b3e-b752-b24a625bacc0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.677732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.679618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:11.681930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.682885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.683800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.684144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] PATH_INFO: `/v3/projects/2ee350b8498441a8bf2a673f66505cc6/users/8de186b0bbf84d678689ac1f3243103c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.695266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960bba-0cc2-4845-b0fd-e6a617049e11 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.700111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.701011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.701296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.701390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee350b8498441a8bf2a673f66505cc6, user_id=8de186b0bbf84d678689ac1f3243103c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.703123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] PATH_INFO: `/v3/users/0ec158e60b2d41ae854536f88f03a9a3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.703123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec158e60b2d41ae854536f88f03a9a3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.703982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.704752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.820968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1029/2061] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:11 2022] PUT /identity/v3/projects/2ee350b8498441a8bf2a673f66505cc6/users/8de186b0bbf84d678689ac1f3243103c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:11.825052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.896575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634a18-4c4b-4f22-9b1a-9931984d5364 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.901285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.902484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.902764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.903041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.903765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.905387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.919018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1030/2062] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:11.920062 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:11.930007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec158e60b2d41ae854536f88f03a9a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:11.930437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758023f-6a30-47ca-abc3-ae0a1dc42858 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:11.935306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:11.936269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.936656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.936944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] PATH_INFO: `/v3/projects/2ee350b8498441a8bf2a673f66505cc6/users/8de186b0bbf84d678689ac1f3243103c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.939060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee350b8498441a8bf2a673f66505cc6, user_id=8de186b0bbf84d678689ac1f3243103c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:11.941732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc188f-ce0a-487a-8b7f-1f892c1fd205 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:11.942402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec158e60b2d41ae854536f88f03a9a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:11.952434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1031/2063] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:11 2022] PUT /identity/v3/projects/2ee350b8498441a8bf2a673f66505cc6/users/8de186b0bbf84d678689ac1f3243103c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:11.963323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:11.963611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:11.963887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:11.994877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] MFA Rules not processed for user `8de186b0bbf84d678689ac1f3243103c`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:11.996215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:12.042097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec158e60b2d41ae854536f88f03a9a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.042843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec158e60b2d41ae854536f88f03a9a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.043455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-17737f48-ee8a-47d9-9313-af66461c7e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec158e60b2d41ae854536f88f03a9a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.062298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1033/2064] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:11 2022] DELETE /identity/v3/users/0ec158e60b2d41ae854536f88f03a9a3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:12.066628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:12.066917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:12.067198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:12.068936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1032/2065] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:12.080040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:12.081612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:12.089110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:12.090330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:12.138841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc476-0308-4edd-8dab-6f282e9c3449 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:12.144372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:12.151112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1034/2066] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:12.160995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:12.162381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:12.190948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:12.191598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:12.191904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:12.192186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:12.224413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:12.237795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ee63a-7f7e-4b59-9c84-e975b04a5320 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:12.241641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:12.242286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:12.242724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:12.242995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:12.254422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:12.256001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:12.256773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:12.258250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:12.266195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1035/2067] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:12.271816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1033/2068] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:12.687050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:12.688652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:12.702032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542274e-6712-49c5-9139-a3535c38d4cd service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:12.706855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:12.707708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:12.707993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:12.708277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] PATH_INFO: `/v3/projects/dbb61f73ff774ec1b2a47cc32beeebec` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:12.709547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb61f73ff774ec1b2a47cc32beeebec)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:12.711676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:12.846215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb61f73ff774ec1b2a47cc32beeebec', 'request_id': 'req-7099447a-52b3-4219-b0ec-16943be2d1a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.854124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb61f73ff774ec1b2a47cc32beeebec', 'request_id': 'req-7099447a-52b3-4219-b0ec-16943be2d1a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.859940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb61f73ff774ec1b2a47cc32beeebec', 'request_id': 'req-7099447a-52b3-4219-b0ec-16943be2d1a3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.888067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:12.971935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] The token cache is being invalidate because project dbb61f73ff774ec1b2a47cc32beeebec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:12.971935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.971935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:12.974540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1036/2069] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:12 2022] DELETE /identity/v3/projects/dbb61f73ff774ec1b2a47cc32beeebec => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:12.977418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:12.988159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bc618-ca7b-434c-a49d-fbbaab4a845b service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:13.048779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:13.049783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:13.050076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:13.050356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] PATH_INFO: `/v3/users/680529e3bbcf46d4a6cfcb83d1e8616c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:13.051621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680529e3bbcf46d4a6cfcb83d1e8616c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:13.053606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:13.231491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:13.349154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680529e3bbcf46d4a6cfcb83d1e8616c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.356787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680529e3bbcf46d4a6cfcb83d1e8616c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.357453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680529e3bbcf46d4a6cfcb83d1e8616c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.447096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680529e3bbcf46d4a6cfcb83d1e8616c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.447794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680529e3bbcf46d4a6cfcb83d1e8616c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.472703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1034/2070] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:12 2022] DELETE /identity/v3/users/680529e3bbcf46d4a6cfcb83d1e8616c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:13.617439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:13.618903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:13.632354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099447a-52b3-4219-b0ec-16943be2d1a3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:13.690371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:13.691420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:13.696460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:13.696460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] PATH_INFO: `/v3/users/e2101751451042b7b9c4c3df826e496e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:13.696460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2101751451042b7b9c4c3df826e496e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:13.696460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:13.813265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:13.923676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2101751451042b7b9c4c3df826e496e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.930523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2101751451042b7b9c4c3df826e496e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:13.986316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:13.987791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:14.003799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626bc83-d922-45e2-90da-39abb76c42a1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:14.008878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:14.009892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:14.010238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:14.010654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] PATH_INFO: `/v3/projects/aaeae6872bcc429399fc34209f233395` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:14.012446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaeae6872bcc429399fc34209f233395)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:14.014417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:14.014651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2101751451042b7b9c4c3df826e496e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.017122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2101751451042b7b9c4c3df826e496e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.022678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b0c3d05-c34f-4143-991b-781bf266bea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2101751451042b7b9c4c3df826e496e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.036709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1037/2071] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:13 2022] DELETE /identity/v3/users/e2101751451042b7b9c4c3df826e496e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:14.041227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:14.041521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:14.041804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:14.060308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:14.061538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:14.121697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1038/2072] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:14.131433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:14.132856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:14.158647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaeae6872bcc429399fc34209f233395', 'request_id': 'req-e3ecc11b-5871-480a-9546-4ef63c2594d5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.165340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaeae6872bcc429399fc34209f233395', 'request_id': 'req-e3ecc11b-5871-480a-9546-4ef63c2594d5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.172604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaeae6872bcc429399fc34209f233395', 'request_id': 'req-e3ecc11b-5871-480a-9546-4ef63c2594d5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.196424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b988-b9fd-4dc1-bb68-7f0b7d35ed3f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:14.226055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:14.247439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:14.248017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:14.248303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:14.248650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:14.256764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] The token cache is being invalidate because project aaeae6872bcc429399fc34209f233395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:14.257288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.258253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.263321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1035/2073] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:13 2022] DELETE /identity/v3/projects/aaeae6872bcc429399fc34209f233395 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:14.267248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:14.268923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:14.316292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:14.318146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:14.329646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1039/2074] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:14.349295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecc11b-5871-480a-9546-4ef63c2594d5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:14.355594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:14.356799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:14.357081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:14.357354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] PATH_INFO: `/v3/users/8ec4486473264adcade99cab42893710` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:14.358897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec4486473264adcade99cab42893710)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:14.360902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:14.595977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec4486473264adcade99cab42893710'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.603091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec4486473264adcade99cab42893710'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.603979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec4486473264adcade99cab42893710'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.659364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:14.716527 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec4486473264adcade99cab42893710'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.717165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec4486473264adcade99cab42893710'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:14.737010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1036/2075] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:14 2022] DELETE /identity/v3/users/8ec4486473264adcade99cab42893710 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:14.809885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:14.811562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:14.833071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0e44f-8ccb-430f-9c0d-3fab5c254f4e service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:14.898192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:14.899098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:14.899378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:14.899659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] PATH_INFO: `/v3/projects/bb0c7b47d3704f5e9b05d45e3595281b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:14.900875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0c7b47d3704f5e9b05d45e3595281b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:14.902831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:15.054710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0c7b47d3704f5e9b05d45e3595281b', 'request_id': 'req-5b3e2487-8761-489d-8d6e-c9967f21a5ee'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.061988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0c7b47d3704f5e9b05d45e3595281b', 'request_id': 'req-5b3e2487-8761-489d-8d6e-c9967f21a5ee'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.068500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0c7b47d3704f5e9b05d45e3595281b', 'request_id': 'req-5b3e2487-8761-489d-8d6e-c9967f21a5ee'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.141966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] The token cache is being invalidate because project bb0c7b47d3704f5e9b05d45e3595281b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:15.142336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.142955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.144821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1040/2076] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:14 2022] DELETE /identity/v3/projects/bb0c7b47d3704f5e9b05d45e3595281b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:15.148625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:15.150078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:15.163603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b729f6d-24a9-496d-afb0-028a10a98820 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:15.182049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:15.225891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:15.226687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.226973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.227253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] PATH_INFO: `/v3/users/8de186b0bbf84d678689ac1f3243103c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.228784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de186b0bbf84d678689ac1f3243103c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:15.233533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:15.295991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:15.301110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:15.321360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e2487-8761-489d-8d6e-c9967f21a5ee admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:15.330603 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:15.331517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.331897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.332915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] PATH_INFO: `/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.334179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a6e1ceab1c40ab9cf528a8f1a16cd8)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:15.336148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:15.489853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a6e1ceab1c40ab9cf528a8f1a16cd8', 'request_id': 'req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.490533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de186b0bbf84d678689ac1f3243103c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.496616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a6e1ceab1c40ab9cf528a8f1a16cd8', 'request_id': 'req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.498446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de186b0bbf84d678689ac1f3243103c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.499122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de186b0bbf84d678689ac1f3243103c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.503537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a6e1ceab1c40ab9cf528a8f1a16cd8', 'request_id': 'req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.587891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de186b0bbf84d678689ac1f3243103c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.588495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8652424f-846e-4709-95db-2b2039a73b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de186b0bbf84d678689ac1f3243103c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.606435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1037/2077] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:15 2022] DELETE /identity/v3/users/8de186b0bbf84d678689ac1f3243103c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:15.661994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] The token cache is being invalidate because project 36a6e1ceab1c40ab9cf528a8f1a16cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:15.662552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.662938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:15.669270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1041/2078] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:15 2022] DELETE /identity/v3/projects/36a6e1ceab1c40ab9cf528a8f1a16cd8 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:15.683637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a9b87-88a2-4f3d-a06b-5a3ebd491344 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.684228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a9b87-88a2-4f3d-a06b-5a3ebd491344 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.686926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a9b87-88a2-4f3d-a06b-5a3ebd491344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.732447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:15.743233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b67a9b87-88a2-4f3d-a06b-5a3ebd491344 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:15.744529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b67a9b87-88a2-4f3d-a06b-5a3ebd491344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:15.831912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:15.840929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:15.841284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1038/2079] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:15.845305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.845305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.845610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.891822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:15.893633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:15.907744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1039/2080] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:15.911799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:15.922540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627c144-fb6a-4641-8c2a-9c32b6647abd None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:15.928935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:15.929793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.930266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.930538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.931660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:15.932367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:15.934884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:15.935127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dbf10-c9fa-4685-ba63-3bef39e86ea9 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:15.942918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:15.943810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.944134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.944408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] PATH_INFO: `/v3/users/0088ea2a4d3b45b19fd80df3a65fecb3` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.947309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0088ea2a4d3b45b19fd80df3a65fecb3)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:15.950010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:15.955807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1040/2081] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:30:15.957493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:15.968198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a083b8-8645-469d-93a3-7b625319ebe2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:15.974083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:15.974791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:15.975431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:15.975431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:15.976077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:15.978112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.119140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1041/2082] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:15 2022] POST /identity/v3/projects => generated 399 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:16.120066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.124229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.197141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f074aa-4c53-4068-a1de-7ee1214a068c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.202519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.203382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.203663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.203940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] PATH_INFO: `/v3/projects/2ee350b8498441a8bf2a673f66505cc6` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.205166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee350b8498441a8bf2a673f66505cc6)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.207270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.244842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0088ea2a4d3b45b19fd80df3a65fecb3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.252306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0088ea2a4d3b45b19fd80df3a65fecb3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.348225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee350b8498441a8bf2a673f66505cc6', 'request_id': 'req-239ff215-3625-440a-84c4-fa1550f6db0c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.353928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee350b8498441a8bf2a673f66505cc6', 'request_id': 'req-239ff215-3625-440a-84c4-fa1550f6db0c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.361067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee350b8498441a8bf2a673f66505cc6', 'request_id': 'req-239ff215-3625-440a-84c4-fa1550f6db0c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.367570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0088ea2a4d3b45b19fd80df3a65fecb3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.368218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0088ea2a4d3b45b19fd80df3a65fecb3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.368872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0088ea2a4d3b45b19fd80df3a65fecb3'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.386616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1042/2083] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:15 2022] DELETE /identity/v3/users/0088ea2a4d3b45b19fd80df3a65fecb3 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:16.389844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.389844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.456344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e33d6c-1e51-48b3-8bc2-e019ba4bf8ff admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.461424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.462099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.463526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:16.463985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.464938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.465693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.467640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.492153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] The token cache is being invalidate because project 2ee350b8498441a8bf2a673f66505cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:16.492443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.493154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-239ff215-3625-440a-84c4-fa1550f6db0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:16.495056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1042/2084] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:16 2022] DELETE /identity/v3/projects/2ee350b8498441a8bf2a673f66505cc6 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:16.496041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a0e-1c27-480c-9aac-ac1105bf8573 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.496327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a0e-1c27-480c-9aac-ac1105bf8573 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.496633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54a0e-1c27-480c-9aac-ac1105bf8573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.515875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bac54a0e-1c27-480c-9aac-ac1105bf8573 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:16.517099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bac54a0e-1c27-480c-9aac-ac1105bf8573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.525070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1043/2085] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:16.528554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.528948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.529227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.548955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:16.550323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.562263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1044/2086] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:16.562974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1043/2087] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:16.563120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.564406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.569701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.572500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.588097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189eeb9-c4c8-4d80-94bc-b51de4f0fbd6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.593918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.594581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.594860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.595133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.595860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.597585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e3b25-1a77-4579-8915-7c6a6f5b08f2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.609092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1044/2088] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:16.610033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.610309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.610582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.631252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:16.631252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.631617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83618-bed1-41e2-8816-2f19fd1753e7 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.635288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.635931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.636763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.637045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.639146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1045/2089] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:16.640011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.646613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.648046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.651811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cb79b-5d76-43d8-994f-158772b4eb88 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.657385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.658122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.658481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.658969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.659939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1045/2090] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:16.660352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.661041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.662249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.671494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7472-141c-4a43-95f1-bc2962bb4b8b service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.677393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1046/2091] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:16.677655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.677655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.677655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.677911 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.678699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:16.678819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.679176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.681364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.690971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c31b-334a-44ff-85a7-c3a2374e59c1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.696238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1046/2092] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:30:16.697113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.705940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.705940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.705940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] PATH_INFO: `/v3/projects/6b8b742dc7ec495887e2152e2e8dfb27/users/0f9dddc40ba34816807b887f30dfd7f5/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.709476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.715336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8b742dc7ec495887e2152e2e8dfb27, user_id=0f9dddc40ba34816807b887f30dfd7f5, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.718226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e151485-e0f0-410a-93f3-4baeda8da3c7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.723890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a0cfa-b300-47b3-9a5a-f63bf9e6b17a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.729147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.729823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.730145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.730424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.731106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.733280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:16.845922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1047/2093] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:16 2022] PUT /identity/v3/projects/6b8b742dc7ec495887e2152e2e8dfb27/users/0f9dddc40ba34816807b887f30dfd7f5/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:16.856056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.856342 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.856646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.883000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] MFA Rules not processed for user `0f9dddc40ba34816807b887f30dfd7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:16.884166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.903438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1047/2094] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:16.904417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.965401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1048/2095] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:16.972639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3bd6-56af-40f5-8a04-83b63e305bda admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:16.980355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:16.987361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:16.991136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:16.991434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:16.991716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:16.992516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:16.994735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:16.997325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.062463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba0c6c-cd7e-48a3-9550-9f6e4e18b6e3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.092029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1048/2096] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:16 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:17.097396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.106882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:17.113556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4aba1-90c7-494c-9127-6d4407fe99ab admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.119846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.121237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.121527 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.121804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.122541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.124163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.131221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.131874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.132160 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.132431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.135369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1049/2097] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:17.139910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.150355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa65c-3cb3-490a-a147-4ac529584b37 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.155630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.156283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.156567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.158088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.158861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.160718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.172621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1050/2098] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:17.175674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.186855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52a14-f71c-4a49-a6df-a425547e3224 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.192574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.193706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.194060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.194406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] PATH_INFO: `/v3/projects/ba05b520a0be4445a854aebd86751c45/users/61a57905381b4353a0842f90a46b36fe/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.197340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.199405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.209096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1049/2099] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:17.211399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba05b520a0be4445a854aebd86751c45, user_id=61a57905381b4353a0842f90a46b36fe, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.214870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48418bd2-40dd-44d8-a447-6c82a4d35b33 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.252969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.254408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:17.284685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e89e-5772-4325-a7c6-02d2c2b0bdbc service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.292409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.293928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.298109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.298351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] PATH_INFO: `/v3/projects/1ae23d2cb99648a89af0efc876c1410e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.305476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae23d2cb99648a89af0efc876c1410e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.313766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.313891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1051/2100] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:17 2022] PUT /identity/v3/projects/ba05b520a0be4445a854aebd86751c45/users/61a57905381b4353a0842f90a46b36fe/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:17.330198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.331831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.331831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.380841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] MFA Rules not processed for user `61a57905381b4353a0842f90a46b36fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:17.387934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:17.459425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1052/2101] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:17.463774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.513385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:17.526383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae23d2cb99648a89af0efc876c1410e', 'request_id': 'req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:17.545325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae23d2cb99648a89af0efc876c1410e', 'request_id': 'req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:17.563356 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae23d2cb99648a89af0efc876c1410e', 'request_id': 'req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:17.611844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0db40-563f-4e8e-8278-0921e1997506 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.664559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] The token cache is being invalidate because project 1ae23d2cb99648a89af0efc876c1410e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:17.665204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:17.665755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:17.671547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1050/2102] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:17 2022] DELETE /identity/v3/projects/1ae23d2cb99648a89af0efc876c1410e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:17.694310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.696444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:17.726384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.728567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.728567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.728567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.728567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.740788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.747695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:17.789741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ef16-8b98-47a3-b90e-cf8f970b7db2 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.795165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.796060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.796343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.796667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] PATH_INFO: `/v3/users/26b6392796bf4f39af67dda942604fd7` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.797841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b6392796bf4f39af67dda942604fd7)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.799761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.854570 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1053/2103] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:17 2022] POST /identity/v3/projects => generated 383 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:17.859272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:17.861006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:17.926451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bd75e-909b-45f9-b005-c511385277cb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:17.931451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:17.932151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:17.932431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:17.932738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:17.933424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:17.935332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:17.950162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:18.003987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b6392796bf4f39af67dda942604fd7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.011258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b6392796bf4f39af67dda942604fd7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.082126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1054/2104] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:17 2022] POST /identity/v3/users => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:18.085983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.096369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda2eb7-14e4-4ffa-a851-8e09ca445bb5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.101128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.101783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.102073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.102355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.103131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.104806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.111181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b6392796bf4f39af67dda942604fd7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.113282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b6392796bf4f39af67dda942604fd7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.114269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b6392796bf4f39af67dda942604fd7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.117116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1055/2105] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:18.124294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.127739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:18.140723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1051/2106] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:17 2022] DELETE /identity/v3/users/26b6392796bf4f39af67dda942604fd7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:18.151688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f54c-56e7-4c5b-a375-fe21ae8166ea admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.218265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.219216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.219529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.219815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] PATH_INFO: `/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6/users/bd4e53581432476f97ef6ad14897e598/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.235907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0354f07e894ca1a396d8e53ddb47d6, user_id=bd4e53581432476f97ef6ad14897e598, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.238684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.397374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1056/2107] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:18 2022] PUT /identity/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6/users/bd4e53581432476f97ef6ad14897e598/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:18.401447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.412613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0e16-442f-4c97-a4c2-58509d5d99e5 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.473275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.474397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.474878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.475323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.476212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.478121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897724-778a-478c-b8d4-5b8aad8010a0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.492269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1052/2108] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:18.496335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.514353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a0e8f-d508-4d9f-be84-8dc830ccf6fd admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.521186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.522573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.523288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.523766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] PATH_INFO: `/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6/users/bd4e53581432476f97ef6ad14897e598/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.526535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0354f07e894ca1a396d8e53ddb47d6, user_id=bd4e53581432476f97ef6ad14897e598, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.530718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.542146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1057/2109] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:18 2022] PUT /identity/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6/users/bd4e53581432476f97ef6ad14897e598/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:18.553724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.554044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.554326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.587030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] MFA Rules not processed for user `bd4e53581432476f97ef6ad14897e598`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:18.588275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:18.659413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1053/2110] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:18.671288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.673085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:18.702392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.703852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:18.736262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c4a39-1cb7-49b4-910e-4b7c6723dada admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.740482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.741181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.741460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.741733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.752143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.753765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.764267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1058/2111] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:18.780529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe915375-32df-465e-8e6c-5f23821ccbe5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.785372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:18.786600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:18.786945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:18.787297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] PATH_INFO: `/v3/projects/2e1172b4d5244754afb2021a177ada9c` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:18.788606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1172b4d5244754afb2021a177ada9c)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:18.791105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:18.942638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:18.950385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1172b4d5244754afb2021a177ada9c', 'request_id': 'req-c6083659-72f6-4840-85a3-09bd971ca8da'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.951864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243814d-391e-4650-8e10-09c32814f303 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:18.959094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1172b4d5244754afb2021a177ada9c', 'request_id': 'req-c6083659-72f6-4840-85a3-09bd971ca8da'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:18.965862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1172b4d5244754afb2021a177ada9c', 'request_id': 'req-c6083659-72f6-4840-85a3-09bd971ca8da'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:19.002476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.003119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.003400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.003677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.006474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.035100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] The token cache is being invalidate because project 2e1172b4d5244754afb2021a177ada9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:19.035100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:19.035391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c6083659-72f6-4840-85a3-09bd971ca8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:19.037292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1054/2112] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:18 2022] DELETE /identity/v3/projects/2e1172b4d5244754afb2021a177ada9c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:19.042008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.042989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.043341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.062179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:19.063381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.079798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.087568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f4f1d-07be-4b3b-8894-95130905eeac service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.090980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1059/2113] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:19.131028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1055/2114] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:19.139633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.139921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.140197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.158823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:19.160018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.169905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1060/2115] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:19.174480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.185035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bb4db-412a-411f-b791-d9eaf28bb45b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.189755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.190449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.190731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.191005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.191784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:19.192513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.194150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.206106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1056/2116] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:30:19.210445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.220106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffaad1c-2e13-4966-b5a8-8d1365fa997f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.226040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.228726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.229187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.229487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.230813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.233125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.350670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1061/2117] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:19 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:19.354528 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.366363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4250f01-6af4-4e3e-9a95-81635fcb502e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.426254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.426922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.427217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.427481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.428192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.430132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.554165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:19.565173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1057/2118] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:19 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:19.566261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.577087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90783ae3-cc09-4d57-b2e2-2e897ecc7628 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.582171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.582826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.583107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.583376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.584247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.585901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.596682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1062/2119] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:19.600714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.611564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f7990-7d11-4829-b5c4-1fcc967d8c75 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.616223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.616905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.617187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.617460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.618178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.619780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.629949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1058/2120] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:19.634510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.645328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659e4e5-90c7-4a3a-be20-8e76fb11a0c6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.650180 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.651102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.651368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.651667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] PATH_INFO: `/v3/projects/5e2262a23d344573b70505e0a33f88ee/users/3edc6cf0ab4148209dbe1333f402a470/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.675108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2262a23d344573b70505e0a33f88ee, user_id=3edc6cf0ab4148209dbe1333f402a470, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.677873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53984b0d-3fe5-4042-a072-2f508a3e19dc admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.728328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.730140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.743152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c433-8728-4375-aaa3-a46b3c43b9cf admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.747971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.748860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.749150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.749424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] PATH_INFO: `/v3/users/61a57905381b4353a0842f90a46b36fe` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.750644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a57905381b4353a0842f90a46b36fe)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.752544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.765543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1063/2121] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:19 2022] PUT /identity/v3/projects/5e2262a23d344573b70505e0a33f88ee/users/3edc6cf0ab4148209dbe1333f402a470/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:19.770802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.771095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.771374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.788679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:19.789848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.819835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:19.855199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1064/2122] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:19.864956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:19.866378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:19.922588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b3120-d0ef-4bc6-a1e0-f864f2967428 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:19.926499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:19.927141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:19.927450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:19.927724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:19.937237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:19.938657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf0e3a-d71f-408b-a859-01d8c847ca45 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:19.948321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1065/2123] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:20.021370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:20.021725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:20.022160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:20.030899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a57905381b4353a0842f90a46b36fe'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:20.038799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a57905381b4353a0842f90a46b36fe'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:20.043493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:20.044995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:20.101943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1066/2124] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:30:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:20.145636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:20.147188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:20.175779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a57905381b4353a0842f90a46b36fe'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:20.176462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a57905381b4353a0842f90a46b36fe'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:20.177175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-71dd483e-a363-4303-9c78-1a6e85425890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a57905381b4353a0842f90a46b36fe'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:20.197936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1059/2125] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:19 2022] DELETE /identity/v3/users/61a57905381b4353a0842f90a46b36fe => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:20.203112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:20.203405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:20.203682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:20.221616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:20.222989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04276291-590c-40b6-846f-b02b98aa4c9a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:20.223443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:20.285648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1060/2126] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:20.299503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:20.302125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:20.302783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:20.303112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:20.303238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:20.303329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:20.305296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:20.378530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:20.382399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:20.419049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1067/2127] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 09:30:20.451652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717f983-7320-4c63-9f02-1c6f40f25da6 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:20.460894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:20.461559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:20.461846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:20.462667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:20.473906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:20.475719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:20.488093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1061/2128] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:21.004041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:21.015841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:21.060876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912ec98-3e47-4ea5-808d-5b9d698f6e0d service swift] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:21.066817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:21.067993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:21.068276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:21.068753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] PATH_INFO: `/v3/projects/ba05b520a0be4445a854aebd86751c45` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:21.070061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba05b520a0be4445a854aebd86751c45)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:21.072100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:21.255237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba05b520a0be4445a854aebd86751c45', 'request_id': 'req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.287558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba05b520a0be4445a854aebd86751c45', 'request_id': 'req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.297663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba05b520a0be4445a854aebd86751c45', 'request_id': 'req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.333855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:21.402523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] The token cache is being invalidate because project ba05b520a0be4445a854aebd86751c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:21.402915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.406209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-1fed866a-3ba7-41ba-b8c1-ca49ebda23a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.409653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1068/2129] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:21 2022] DELETE /identity/v3/projects/ba05b520a0be4445a854aebd86751c45 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:21.411927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:21.433733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cb38-dee0-4aee-9e13-45450ec012f9 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:21.521999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:21.545494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:21.546372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:21.546914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:21.547398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] PATH_INFO: `/v3/users/bd4e53581432476f97ef6ad14897e598` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:21.548976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4e53581432476f97ef6ad14897e598)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:21.551083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:21.717213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:21.772105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4e53581432476f97ef6ad14897e598'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.789486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4e53581432476f97ef6ad14897e598'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.805559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f7a9-7a82-4a14-bca0-294ee2c1170a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:21.806195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f7a9-7a82-4a14-bca0-294ee2c1170a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:21.807886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f7a9-7a82-4a14-bca0-294ee2c1170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:21.843899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8ef5f7a9-7a82-4a14-bca0-294ee2c1170a None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:21.845971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8ef5f7a9-7a82-4a14-bca0-294ee2c1170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:21.908025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4e53581432476f97ef6ad14897e598'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.910264 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4e53581432476f97ef6ad14897e598'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.912073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4e53581432476f97ef6ad14897e598'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:21.916905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1069/2130] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:30:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:21.918914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:21.922400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:21.922400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:21.952996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] MFA Rules not processed for user `3edc6cf0ab4148209dbe1333f402a470`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:21.954270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:21.955107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1062/2131] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:21 2022] DELETE /identity/v3/users/bd4e53581432476f97ef6ad14897e598 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:21.957810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:21.965655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.085216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1070/2132] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:22.089710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.091184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.092001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b405e-0127-4de9-bbe6-97daf3537b26 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.097357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.098005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.098314 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.098610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.100516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.170690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4d2b-e089-4fec-a8fc-fbbefddb164a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.176470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.177217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.177501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.177776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.178510 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.181285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.184995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.187624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.202041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1063/2133] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:30:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:30:22.326541 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1071/2134] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:22 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:22.330681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.341784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4b1b-de84-4a64-a235-3e7bb49b4b71 service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.398093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.398756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.399148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.399455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.400159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.402631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.424147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.425886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.438794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23416-d9fe-4138-9666-952732c0b231 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.443587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.444443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.444769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.445053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] PATH_INFO: `/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.446250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0354f07e894ca1a396d8e53ddb47d6)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.448168 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.528335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1064/2135] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:22 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:22.532567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.543605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447eae1-1bc1-4600-a73e-743502cdc3e7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.548544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.549239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.549523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.549801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.550533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.552878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.553935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:22.565211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1065/2136] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:22.569209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.580140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a78af-d4e5-4772-815d-2c6a7c293b20 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.585505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.586384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.586796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.587075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] PATH_INFO: `/v3/projects/74a84d30ec034704a575b868875be876/users/e499df3d5ab248d59d0f7e391c770662/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.602865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a84d30ec034704a575b868875be876, user_id=e499df3d5ab248d59d0f7e391c770662, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.605678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.708302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1066/2137] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:22 2022] PUT /identity/v3/projects/74a84d30ec034704a575b868875be876/users/e499df3d5ab248d59d0f7e391c770662/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:22.716948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0354f07e894ca1a396d8e53ddb47d6', 'request_id': 'req-77b80b8e-d6fe-46df-a2bc-800557c8119b'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:22.720846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.723101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0354f07e894ca1a396d8e53ddb47d6', 'request_id': 'req-77b80b8e-d6fe-46df-a2bc-800557c8119b'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:22.730649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0354f07e894ca1a396d8e53ddb47d6', 'request_id': 'req-77b80b8e-d6fe-46df-a2bc-800557c8119b'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:22.795061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855802c8-5fa7-4429-84b6-2cc8b1827812 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.846746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] The token cache is being invalidate because project 9e0354f07e894ca1a396d8e53ddb47d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:22.847029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:22.847649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-77b80b8e-d6fe-46df-a2bc-800557c8119b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:22.849263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1072/2138] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:22 2022] DELETE /identity/v3/projects/9e0354f07e894ca1a396d8e53ddb47d6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:22.849880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.850548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.850832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.851110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.851917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.853554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.855303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64a9d7-47e0-407e-a383-0222f67af366 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.855590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64a9d7-47e0-407e-a383-0222f67af366 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.855860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64a9d7-47e0-407e-a383-0222f67af366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.864714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1067/2139] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:30:22.868954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:22.871588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.875514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9d64a9d7-47e0-407e-a383-0222f67af366 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:22.876714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9d64a9d7-47e0-407e-a383-0222f67af366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.885016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512fcee-5608-485a-9eed-65240a9dc20f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:22.891414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:22.892250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1073/2140] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:22.892367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.892557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.892864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] PATH_INFO: `/v3/projects/74a84d30ec034704a575b868875be876/users/e499df3d5ab248d59d0f7e391c770662/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.894948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a84d30ec034704a575b868875be876, user_id=e499df3d5ab248d59d0f7e391c770662, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:22.897642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:22.900143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:22.900435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:22.900754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:22.910626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1068/2141] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:22 2022] PUT /identity/v3/projects/74a84d30ec034704a575b868875be876/users/e499df3d5ab248d59d0f7e391c770662/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:22.920902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:22.922269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:22.985333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1074/2142] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:22.989740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.002353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ff5e-7dff-4a56-bdcb-d52742cba137 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.007341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.007968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.008246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.008516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.009746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:23.009746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.011315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843be14-64da-42e1-8baf-ce254834a513 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.028476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1069/2143] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:30:23.033707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.050655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4eea3-ff3a-423f-9592-a4b193069a25 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.057862 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.058819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.059276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.059620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.060542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.062634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.085454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.085801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.086107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.105729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:23.106932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:23.161271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1070/2144] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:30:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:23.173643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.174978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:23.244088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a73c-8749-4ad6-895a-29e8326c840a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.248390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.249067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.249347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.249618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.257848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.259427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.267585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1071/2145] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 09:30:23.279615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1075/2146] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:23 2022] POST /identity/v3/projects => generated 363 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:23.280561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.288414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:23.293522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35152b-80e3-43d4-956e-47296016befd service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.351544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.352337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.352682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.352961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.353643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.356253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.462142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1072/2147] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:23 2022] POST /identity/v3/users => generated 282 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:30:23.466828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.477290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e157cd-4aaa-4b80-9897-0c176201003d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.482226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.483550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.483550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.483744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.484462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.486082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.496940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1076/2148] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:23.500906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.512498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fa98-5fa8-43ef-915f-0a70bc960309 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.517573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.518267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.518949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.518949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.519562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.521181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5abeba-4e51-48ea-af44-6e878c1ad090 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.534353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1073/2149] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:23.538341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:23.547438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:23.551993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1c7d3-f849-4da5-88ea-06f1893c1ac5 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:23.557892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:23.558776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.559056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.559340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] PATH_INFO: `/v3/projects/c6397a680b434385b0b8adc2c1689593/users/4054611717a941a59e88770881cce36e/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.579730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6397a680b434385b0b8adc2c1689593, user_id=4054611717a941a59e88770881cce36e, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:23.582518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f595b0-46e0-452b-8140-5e42c9e02679 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:23.908169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1077/2150] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:23 2022] PUT /identity/v3/projects/c6397a680b434385b0b8adc2c1689593/users/4054611717a941a59e88770881cce36e/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:23.919089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.919385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.919385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.941789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:23.942079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:23.942383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:23.949417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] MFA Rules not processed for user `4054611717a941a59e88770881cce36e`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:23.950712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:23.960252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:23.961449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:24.018472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1078/2151] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:30:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:24.024179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1074/2152] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:24.032327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:24.036180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:24.037586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:24.102192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945d4e4-cbc5-4a53-9b4f-552044a8eada None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:24.108173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:24.109148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:24.109433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:24.109707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:24.115402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2d31d-67f6-448b-8e9c-83568f2f7947 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:24.119381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:24.121397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fc4c1-cd8f-4286-8fd6-65d9f930b0f6 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:24.122875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:24.124146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:24.124438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:24.124778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:24.131187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1075/2153] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:24.135173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:24.137752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:24.147893 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1079/2154] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:30:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:30:24.248051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:24.536377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:25.054806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:25.055349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:25.055668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:25.107663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] MFA Rules not processed for user `e499df3d5ab248d59d0f7e391c770662`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:25.111345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:25.195323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1076/2155] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:25.208841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:25.233364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deeaaa-d75d-4c76-9c63-28258a0c1535 service swift] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:25.237583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:25.238547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:25.238993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:25.239341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:25.241607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:25.302380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:25.334628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:25.336978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:25.350638 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1080/2156] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:25.565554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:26.708021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:26.862480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:27.362087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:27.467505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:27.978559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:28.281416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:28.291134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235db4d4-59ed-4f78-a5c0-7dbff5bb0a67 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:28.294978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:28.295726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:28.296053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:28.296361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:28.305482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:28.306901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:28.316832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1077/2157] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:28.401480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:28.403022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:28.473287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86716132-ade3-4bf7-a9dc-f6dd1232c61f service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:28.477995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:28.478850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:28.479147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:28.479446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] PATH_INFO: `/v3/users/7e44a60fc7244d6d88a20aeac53c3353` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:28.480941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e44a60fc7244d6d88a20aeac53c3353)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:28.483209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:28.489396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:28.653520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:28.744700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e44a60fc7244d6d88a20aeac53c3353'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:28.752243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e44a60fc7244d6d88a20aeac53c3353'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:28.752958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e44a60fc7244d6d88a20aeac53c3353'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:28.802947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:28.804475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:28.818578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4e601-12b9-4040-8eed-e1921d32945e service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:28.825011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:28.825962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:28.826338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:28.826579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] PATH_INFO: `/v3/users/0f9dddc40ba34816807b887f30dfd7f5` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:28.827821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9dddc40ba34816807b887f30dfd7f5)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:28.829888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:28.856795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e44a60fc7244d6d88a20aeac53c3353'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:28.857445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f5ca33ee-fba9-46b2-b462-60c1b0b79e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e44a60fc7244d6d88a20aeac53c3353'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:28.876465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1081/2158] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:28 2022] DELETE /identity/v3/users/7e44a60fc7244d6d88a20aeac53c3353 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:28.880689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:28.880980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:28.881258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:28.898245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:28.899504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:28.960252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1082/2159] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:28.970461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:28.971819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:29.029846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a216f14-5105-4e6f-a6bd-93334c4fe799 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:29.033955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:29.034601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:29.034880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:29.035153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:29.044221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:29.045683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562486-2eef-42e6-b849-33b808c3bc35 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:29.055159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1083/2160] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:30:29.083632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9dddc40ba34816807b887f30dfd7f5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.090805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9dddc40ba34816807b887f30dfd7f5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.182003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9dddc40ba34816807b887f30dfd7f5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.183227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9dddc40ba34816807b887f30dfd7f5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.183886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9dddc40ba34816807b887f30dfd7f5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.204507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1078/2161] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:28 2022] DELETE /identity/v3/users/0f9dddc40ba34816807b887f30dfd7f5 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:29.209069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:29.212244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:29.212244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:29.228061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:29.229311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:29.254465 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:29.295756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1084/2162] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:30:29.305685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:29.307129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:29.334806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:29.337244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:29.355113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781cdbb-2285-4081-a032-52d519f6d881 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:29.363098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:29.363971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:29.364349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:29.364526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] PATH_INFO: `/v3/projects/44089f44ea384df88ea8e9411dc94476` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:29.365819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44089f44ea384df88ea8e9411dc94476)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:29.367808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:29.373081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e0387-97d3-4069-81c9-962ce13b31d1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:29.377079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:29.377704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:29.377989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:29.378307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:29.390006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:29.390232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:29.404814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1079/2163] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:29.471124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44089f44ea384df88ea8e9411dc94476', 'request_id': 'req-85c5edbf-0e37-4cc7-b185-a35d66c08674'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.477548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44089f44ea384df88ea8e9411dc94476', 'request_id': 'req-85c5edbf-0e37-4cc7-b185-a35d66c08674'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.484957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44089f44ea384df88ea8e9411dc94476', 'request_id': 'req-85c5edbf-0e37-4cc7-b185-a35d66c08674'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.578800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] The token cache is being invalidate because project 44089f44ea384df88ea8e9411dc94476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:29.579090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.579739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-85c5edbf-0e37-4cc7-b185-a35d66c08674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:29.581360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1085/2164] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:29 2022] DELETE /identity/v3/projects/44089f44ea384df88ea8e9411dc94476 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:29.865553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:29.867022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:29.880286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82bf5-9a03-4d9c-89db-7be402766492 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:29.940463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:29.941285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:29.941572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:29.941851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] PATH_INFO: `/v3/projects/6b8b742dc7ec495887e2152e2e8dfb27` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:29.943181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8b742dc7ec495887e2152e2e8dfb27)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:29.945151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:29.946609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:30.087351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8b742dc7ec495887e2152e2e8dfb27', 'request_id': 'req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:30.094241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8b742dc7ec495887e2152e2e8dfb27', 'request_id': 'req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:30.100033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8b742dc7ec495887e2152e2e8dfb27', 'request_id': 'req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:30.201557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] The token cache is being invalidate because project 6b8b742dc7ec495887e2152e2e8dfb27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:30.201871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:30.202582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-341f01fb-2bc1-4fed-b1ad-0979a7b77c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:30.204258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1080/2165] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:29 2022] DELETE /identity/v3/projects/6b8b742dc7ec495887e2152e2e8dfb27 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:39.575212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:40.550326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:52.564063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:52.564564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:52.564789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:52.566397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1086/2166] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:30:52.599176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:52.599662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:52.599892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:52.669173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] MFA Rules not processed for user `6459963216fe4a039289498f1fd8af54`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:52.670874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:52.754618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1081/2167] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:30:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:52.769179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:52.775864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:52.823914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96119-f823-4aa2-b3d4-5727b45920e8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:52.829223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:52.829858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:52.830699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:52.830979 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:52.839973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:52.841643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:52.853182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1087/2168] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:30:53.472786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:53.474248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:53.548503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf0d0-f80e-49ff-b4e3-ed2cbc22ffae None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:53.553102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:53.553988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:53.554273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:53.554551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] PATH_INFO: `/v3/users/3edc6cf0ab4148209dbe1333f402a470` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:53.555820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edc6cf0ab4148209dbe1333f402a470)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:53.557966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:53.731535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edc6cf0ab4148209dbe1333f402a470'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:53.733087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:53.739312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edc6cf0ab4148209dbe1333f402a470'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:53.829566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edc6cf0ab4148209dbe1333f402a470'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:53.830198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edc6cf0ab4148209dbe1333f402a470'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:53.830814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edc6cf0ab4148209dbe1333f402a470'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:53.850095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1082/2169] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:53 2022] DELETE /identity/v3/users/3edc6cf0ab4148209dbe1333f402a470 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:54.151349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:54.164094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bac9-1d00-43b1-9605-dede2e57faf1 service placement] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:54.221780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:54.222639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:54.222921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:54.223197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] PATH_INFO: `/v3/projects/5e2262a23d344573b70505e0a33f88ee` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:54.224373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2262a23d344573b70505e0a33f88ee)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:54.226316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:54.379695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2262a23d344573b70505e0a33f88ee', 'request_id': 'req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.385194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2262a23d344573b70505e0a33f88ee', 'request_id': 'req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.391242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2262a23d344573b70505e0a33f88ee', 'request_id': 'req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.478164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] The token cache is being invalidate because project 5e2262a23d344573b70505e0a33f88ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:54.478522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.479210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.480890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1088/2170] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:54 2022] DELETE /identity/v3/projects/5e2262a23d344573b70505e0a33f88ee => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:54.485851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:54.487243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:54.500076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c35098-4a88-4355-b686-0a9d3edab4a4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:54.554886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:54.555741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:54.556047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:54.556323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] PATH_INFO: `/v3/users/e499df3d5ab248d59d0f7e391c770662` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:54.557521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e499df3d5ab248d59d0f7e391c770662)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:54.559428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:54.965055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e499df3d5ab248d59d0f7e391c770662'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:54.972609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e499df3d5ab248d59d0f7e391c770662'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.044529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e499df3d5ab248d59d0f7e391c770662'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.045227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e499df3d5ab248d59d0f7e391c770662'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.045871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-070c517c-fe3d-4b99-aa78-195e769c7410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e499df3d5ab248d59d0f7e391c770662'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.063916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1083/2171] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:30:54 2022] DELETE /identity/v3/users/e499df3d5ab248d59d0f7e391c770662 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:30:55.454587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:55.456085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:55.469576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d0cfc-f29b-4be0-9cf4-2638bee5ee69 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:55.526271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:55.531156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:55.531156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:55.531156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] PATH_INFO: `/v3/projects/74a84d30ec034704a575b868875be876` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:55.531156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a84d30ec034704a575b868875be876)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:55.531497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:55.553045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:55.690702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a84d30ec034704a575b868875be876', 'request_id': 'req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.696365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a84d30ec034704a575b868875be876', 'request_id': 'req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.703460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a84d30ec034704a575b868875be876', 'request_id': 'req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.805954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] The token cache is being invalidate because project 74a84d30ec034704a575b868875be876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:30:55.806247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.806888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23bd5c9c-ff7d-47fc-b25a-d605c3108ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:30:55.808449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1089/2172] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:30:55 2022] DELETE /identity/v3/projects/74a84d30ec034704a575b868875be876 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:55.813492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:55.813787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:55.814070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:55.833584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:55.834781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:55.880532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:30:55.900686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1084/2173] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:55.909317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:55.909606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:55.909885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:55.930385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:55.936231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:55.940523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1090/2174] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:55.945034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:55.955963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beeaa2-2e60-4008-b718-d3712dcc2e73 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:55.960783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:55.961540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:55.961824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:55.962099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:55.962831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:30:55.963138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:55.964890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:55.977146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1085/2175] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:30:55.985509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:55.994648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d0ce0-357a-4227-a843-f8957600f800 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:55.999802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.000462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.000780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.001071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.001775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.003782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.158073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1091/2176] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:30:55 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:56.162282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:56.172535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ad339-455f-4313-89eb-6c1dd38c6704 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:56.228374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.229005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.229313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.229588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.230283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.232217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.329946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1086/2177] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:30:56 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:30:56.334230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:56.346558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd8d0-99bf-40a4-be61-736aac368e0a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:56.349887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.350544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.350821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.351095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.351849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.353461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.364833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1092/2178] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:56.368786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:56.379423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a638645-5c58-4403-b6ab-f3a0a28b2366 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:56.383993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.384673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.384959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.385230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.385955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.387627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec562b-146c-42c8-b38e-77fe0d76d496 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.398665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1087/2179] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:30:56.402891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:56.413874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336635-4dd9-4ab7-a17d-b09b99d0642d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:56.424003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.425332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.425617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.425896 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] PATH_INFO: `/v3/projects/73bb19e3847744af88746a3e644ba1a1/users/5d80b399c2b64b2f959b62eac2abcbed/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.442260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bb19e3847744af88746a3e644ba1a1, user_id=5d80b399c2b64b2f959b62eac2abcbed, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.444958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.546757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1093/2180] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:30:56 2022] PUT /identity/v3/projects/73bb19e3847744af88746a3e644ba1a1/users/5d80b399c2b64b2f959b62eac2abcbed/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:30:56.552355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311043ff-dc8d-46dc-9588-5927793911c8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.552674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311043ff-dc8d-46dc-9588-5927793911c8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.552958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311043ff-dc8d-46dc-9588-5927793911c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.580034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-311043ff-dc8d-46dc-9588-5927793911c8 None None] MFA Rules not processed for user `5d80b399c2b64b2f959b62eac2abcbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:30:56.581321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-311043ff-dc8d-46dc-9588-5927793911c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:56.646567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1088/2181] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:30:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:30:56.657783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:30:56.659714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:30:56.743626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1562-2a1e-430e-8790-593d4e1c03aa admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:30:56.747953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:30:56.748603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:30:56.748959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:30:56.749236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:30:56.757730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:30:56.759181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:30:56.767907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1094/2182] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:30:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:31:01.971373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8df5c-97ad-445a-91df-7b781f2cca80 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:01.978523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8df5c-97ad-445a-91df-7b781f2cca80 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:01.978523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8df5c-97ad-445a-91df-7b781f2cca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:01.989623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3ca8df5c-97ad-445a-91df-7b781f2cca80 None None] MFA Rules not processed for user `75f6d2e28f024875bda0b20f751bdf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:01.990812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3ca8df5c-97ad-445a-91df-7b781f2cca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:02.025579 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:02.046345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1089/2183] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:02.201082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:02.233080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:02.269799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a5b53-4030-4460-9143-51c41eb8a9c9 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:02.277809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:02.278704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:02.278993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:02.279274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] PATH_INFO: `/v3/users/5d80b399c2b64b2f959b62eac2abcbed` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:02.280526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d80b399c2b64b2f959b62eac2abcbed)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:02.282477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:02.452097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d80b399c2b64b2f959b62eac2abcbed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:02.459329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d80b399c2b64b2f959b62eac2abcbed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:02.460014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d80b399c2b64b2f959b62eac2abcbed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:02.549919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d80b399c2b64b2f959b62eac2abcbed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:02.550615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d80b399c2b64b2f959b62eac2abcbed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:02.569531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1095/2184] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:31:02 2022] DELETE /identity/v3/users/5d80b399c2b64b2f959b62eac2abcbed => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:31:02.574033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:02.574317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:02.574592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:02.594721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:02.596748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:02.656548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1090/2185] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:02.666972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:02.668393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:02.729313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40678-c96f-46cb-aabc-d8a73de31275 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:02.738375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:02.739050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:02.739901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:02.740178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:02.762562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:02.764007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bf85b-3dd3-402a-85e7-75490bf4251a service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:02.775644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1096/2186] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:31:03.300575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:03.302048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:03.315605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50a26c-4839-4e5d-8936-3ba0c02c1e94 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:03.321441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:03.322483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:03.322909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:03.323328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] PATH_INFO: `/v3/projects/73bb19e3847744af88746a3e644ba1a1` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:03.324698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bb19e3847744af88746a3e644ba1a1)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:03.326837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:03.525078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bb19e3847744af88746a3e644ba1a1', 'request_id': 'req-3d648168-82b1-4741-8dd2-492129349f04'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:03.531862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bb19e3847744af88746a3e644ba1a1', 'request_id': 'req-3d648168-82b1-4741-8dd2-492129349f04'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:03.537315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bb19e3847744af88746a3e644ba1a1', 'request_id': 'req-3d648168-82b1-4741-8dd2-492129349f04'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:03.632656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] The token cache is being invalidate because project 73bb19e3847744af88746a3e644ba1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:31:03.632888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:03.633512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3d648168-82b1-4741-8dd2-492129349f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:03.635064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1091/2187] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:31:03 2022] DELETE /identity/v3/projects/73bb19e3847744af88746a3e644ba1a1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:31:03.639932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:03.640398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:03.640571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:03.658260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:03.659451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:03.703605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:03.718641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1097/2188] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:03.726503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:03.726795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:03.727076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:03.745905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:03.747148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:03.756022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1092/2189] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:03.760461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:03.770881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bea4d5-d131-48db-87a9-d9491ef67774 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:03.775806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:03.776458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:03.776776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:03.777057 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:03.777824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:31:03.778141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:03.779768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:03.791872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1098/2190] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:31:03.796976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:03.806125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a2862-6d40-4b6e-a89e-a5238090ab97 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:03.811740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:03.812377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:03.812716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:03.812989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:03.813685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:03.815565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:03.951601 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1093/2191] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:31:03 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:31:03.956422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:03.966482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dc621-029b-4e30-8ad5-bbdd88ba77cf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:04.027073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:04.027746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.028031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.028360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.029092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:04.031030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:04.125371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1099/2192] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:31:03 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:31:04.129448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:04.140033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a53da-aee2-4b8d-8efd-21eff2c6b1a1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:04.144782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:04.145420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.145705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.145981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.146733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:04.148359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:04.159838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1094/2193] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:31:04.163608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:04.174018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562f40-4d0a-4ed5-9dd4-ba89e81dce2c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:04.178646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:04.179290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.179569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.179842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.180559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:04.182158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9cd2a-f784-43d7-975f-6f06b1a146bb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:04.191941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1100/2194] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:31:04.195993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:04.206315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8a7b-78eb-44c6-b5e8-68521a2a9e33 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:04.211248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:04.212920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.212920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.212920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] PATH_INFO: `/v3/projects/76e471342cb24c918197b538b2c8642a/users/7ae6107c814c4bb5a36ba9e80ef875f4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.227896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e471342cb24c918197b538b2c8642a, user_id=7ae6107c814c4bb5a36ba9e80ef875f4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:04.230534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:04.324834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1095/2195] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:31:04 2022] PUT /identity/v3/projects/76e471342cb24c918197b538b2c8642a/users/7ae6107c814c4bb5a36ba9e80ef875f4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:31:04.329888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01ba65-e872-4f37-a890-f6fcd3671e21 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.330182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01ba65-e872-4f37-a890-f6fcd3671e21 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.330463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01ba65-e872-4f37-a890-f6fcd3671e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.348443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-5b01ba65-e872-4f37-a890-f6fcd3671e21 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:04.349673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5b01ba65-e872-4f37-a890-f6fcd3671e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:04.413227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1101/2196] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:31:04.424266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:04.428752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:04.432901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:04.483640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f3c0-0202-445e-bb8c-e2dec0d8a9d7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:04.487339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:04.487982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:04.488262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:04.488536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:04.497565 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:04.499017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:04.508319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1096/2197] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:31:06.105861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d1dd-f2a6-4154-b4e1-1a3a4c263b53 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:06.106412 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d1dd-f2a6-4154-b4e1-1a3a4c263b53 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:06.106616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d1dd-f2a6-4154-b4e1-1a3a4c263b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:06.166311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bd78d1dd-f2a6-4154-b4e1-1a3a4c263b53 None None] MFA Rules not processed for user `7ae6107c814c4bb5a36ba9e80ef875f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:06.168283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bd78d1dd-f2a6-4154-b4e1-1a3a4c263b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:06.245609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:06.264686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1102/2198] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:06.277882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:06.279282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:06.348633 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49930c4f-7665-42ff-bef0-8d95f1278b7b service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:06.352222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:06.352905 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:06.353191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:06.353467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:06.362878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:06.364299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:06.373857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1097/2199] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:31:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:31:09.203523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bd359-8437-4511-9872-d634d8205829 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:09.204049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bd359-8437-4511-9872-d634d8205829 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:09.204146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bd359-8437-4511-9872-d634d8205829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:09.221558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-292bd359-8437-4511-9872-d634d8205829 None None] MFA Rules not processed for user `7ae6107c814c4bb5a36ba9e80ef875f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:09.222806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-292bd359-8437-4511-9872-d634d8205829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:09.230130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1103/2200] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:16.369506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=50 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:19.215969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=10 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:22.955086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:22.956941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:22.970582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f6d9-1256-4b5a-9797-42b25c8a8eb9 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:22.975181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:22.976040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:22.976324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:22.976627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] PATH_INFO: `/v3/users/4054611717a941a59e88770881cce36e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:22.977808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4054611717a941a59e88770881cce36e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:22.986812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:23.143900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4054611717a941a59e88770881cce36e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:23.150792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4054611717a941a59e88770881cce36e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:23.241252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4054611717a941a59e88770881cce36e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:23.242031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4054611717a941a59e88770881cce36e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:23.242694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4054611717a941a59e88770881cce36e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:23.266062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1098/2201] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:31:22 2022] DELETE /identity/v3/users/4054611717a941a59e88770881cce36e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:31:23.270781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:23.271384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:23.271858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:23.291763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:23.293357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:23.361731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1104/2202] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:23.371286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:23.372839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:23.437592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793e2d-0333-44f8-a71e-f614a690e20f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:23.441521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:23.442174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:23.442515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:23.442917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:23.453018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:23.454406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:23.464650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1099/2203] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:31:23.942399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:23.943834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:23.957452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d88b4f-05d7-4579-b40b-26cdf0fe4cd6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:23.960508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:23.961439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:23.962721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:23.963017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] PATH_INFO: `/v3/projects/c6397a680b434385b0b8adc2c1689593` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:23.964270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6397a680b434385b0b8adc2c1689593)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:23.967475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:24.165139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6397a680b434385b0b8adc2c1689593', 'request_id': 'req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:24.170943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6397a680b434385b0b8adc2c1689593', 'request_id': 'req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:24.177111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6397a680b434385b0b8adc2c1689593', 'request_id': 'req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:24.252021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] The token cache is being invalidate because project c6397a680b434385b0b8adc2c1689593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:31:24.252369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:24.253125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ae615e8a-3cdf-45de-88f2-7982ec6dbde9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:31:24.254965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1105/2204] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:31:23 2022] DELETE /identity/v3/projects/c6397a680b434385b0b8adc2c1689593 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:31:31.687034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:31:31.688489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:31.758931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe60bc8-8f97-4d02-90e5-ffe72b766932 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:31:31.771100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:31.818665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:31:31.819340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:31.819668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:31.819951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:31.821760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:31.833035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:31:31.834611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99747464-056e-4724-9dd5-28d5fb9a3c35 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:31:31.845327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1100/2205] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:31:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:31:33.427696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6212-ec48-4a6f-b40f-65b6f14e2063 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:33.428194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6212-ec48-4a6f-b40f-65b6f14e2063 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:33.428308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6212-ec48-4a6f-b40f-65b6f14e2063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:33.445557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-51ae6212-ec48-4a6f-b40f-65b6f14e2063 None None] MFA Rules not processed for user `7ae6107c814c4bb5a36ba9e80ef875f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:33.446746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-51ae6212-ec48-4a6f-b40f-65b6f14e2063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:33.500219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1106/2206] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:31:41.877957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=24 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:43.494016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:31:55.814496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:31:55.815015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:31:55.815135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:31:55.831970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] MFA Rules not processed for user `2f32356bb0bd4234ba817db581b62aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:31:55.833191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:31:55.888741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1101/2207] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:31:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:03.749305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836b1a-b7a6-43c0-abcb-4455a0afcce4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:03.749821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836b1a-b7a6-43c0-abcb-4455a0afcce4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:03.749909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836b1a-b7a6-43c0-abcb-4455a0afcce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:03.800551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-69836b1a-b7a6-43c0-abcb-4455a0afcce4 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:03.801800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-69836b1a-b7a6-43c0-abcb-4455a0afcce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:03.874313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1107/2208] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:32:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:04.038246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:04.039970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:04.144420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65ef09-236f-4d15-ba5e-d345014bad22 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:04.152673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:04.153169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:04.153456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:04.153736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:04.163324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:04.165343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:04.174401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1102/2209] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:32:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:32:13.871313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=43 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:14.170525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=66 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:17.215184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272ac1-3c82-481b-a25c-bc50e4956a9a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:17.217004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272ac1-3c82-481b-a25c-bc50e4956a9a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:17.217763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272ac1-3c82-481b-a25c-bc50e4956a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:17.258122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f9272ac1-3c82-481b-a25c-bc50e4956a9a None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:17.258820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f9272ac1-3c82-481b-a25c-bc50e4956a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:17.277616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1108/2210] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:32:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:17.291147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:17.309270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2adca-76d6-4a02-bb38-51de1361c833 service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:17.324798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:17.324798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:17.324798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:17.324798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:17.334347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:17.335973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:17.346159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1103/2211] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:32:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:32:21.100662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:21.101221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:21.101221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:21.127374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:21.128683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:21.136544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1109/2212] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:32:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:21.148733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:21.162331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03b7f-cc82-4244-96fb-c6bef5a2ecb9 service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:21.169120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:21.169928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:21.170327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:21.170672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:21.182890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:21.184779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:21.195506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1104/2213] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:32:24.829301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:24.834652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:24.935159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a910-c5eb-4916-976c-74bf4ba27abf None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:24.941408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:24.942496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:24.942972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:24.943153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] PATH_INFO: `/v3/users/7ae6107c814c4bb5a36ba9e80ef875f4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:24.945273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae6107c814c4bb5a36ba9e80ef875f4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:24.947832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:25.178890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae6107c814c4bb5a36ba9e80ef875f4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.186236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae6107c814c4bb5a36ba9e80ef875f4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.186875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae6107c814c4bb5a36ba9e80ef875f4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.269542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae6107c814c4bb5a36ba9e80ef875f4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.271382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b8351fbd-c8e8-4694-8dac-0f80c8e1c935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae6107c814c4bb5a36ba9e80ef875f4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.297748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1110/2214] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:32:24 2022] DELETE /identity/v3/users/7ae6107c814c4bb5a36ba9e80ef875f4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:32:25.621134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:25.633876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6bdd-810a-4241-967b-853294149103 service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:25.732792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:25.733945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:25.734404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:25.734683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] PATH_INFO: `/v3/projects/76e471342cb24c918197b538b2c8642a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:25.736612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e471342cb24c918197b538b2c8642a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:25.738515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:25.851083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e471342cb24c918197b538b2c8642a', 'request_id': 'req-4667d039-073c-4843-aa19-97c7a3aad63b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.858346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e471342cb24c918197b538b2c8642a', 'request_id': 'req-4667d039-073c-4843-aa19-97c7a3aad63b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.866044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e471342cb24c918197b538b2c8642a', 'request_id': 'req-4667d039-073c-4843-aa19-97c7a3aad63b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.932629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] The token cache is being invalidate because project 76e471342cb24c918197b538b2c8642a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:32:25.932629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.933090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4667d039-073c-4843-aa19-97c7a3aad63b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:32:25.934739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1105/2215] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:32:25 2022] DELETE /identity/v3/projects/76e471342cb24c918197b538b2c8642a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:32:25.940307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:25.946740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:25.946740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:25.961399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:25.962656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:26.046621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1111/2216] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:26.053884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.054179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.054480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.095768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:26.097287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:26.107412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1106/2217] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:32:26.111452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.128280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffd2c7-2895-4829-8481-f7e2aaeb0439 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.144183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.144986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.145692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.145692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.146478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:32:26.147259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.148493 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.150314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:26.173620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1112/2218] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:32:26.178795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.195076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332554a9-5e64-4b5e-ac1d-fafb1965a60d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.205267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.206242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.206827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.208372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.209462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.212159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.315515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1107/2219] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:32:26 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:32:26.319790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.331477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cbcd0-fdaa-4e34-acff-c1a4ed6f19b7 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.398989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.399830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.400218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.400651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.401447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.403826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.491501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1113/2220] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:32:26 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:32:26.495808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.508682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fde5be-14ac-4625-9f33-14596b134b4f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.514416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.514416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.514690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.514950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.515680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.518045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.532622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1108/2221] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:32:26.535724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.546791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03580d-36f0-4f19-aafb-f6b25ee4ab86 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.552162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.553082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.553464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.553951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.554780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.556535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93555-aba1-4660-8eb0-65cf9631df7d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.572525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1114/2222] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:32:26.575652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.587656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b08fe-6c2b-4f32-80ef-496dbd619c68 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.592568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.593510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.593792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.594177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] PATH_INFO: `/v3/projects/b247e55a5f9e48cb88496964f6037ec7/users/135db966350b4e9ebb29da6eed2dab8b/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.611968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b247e55a5f9e48cb88496964f6037ec7, user_id=135db966350b4e9ebb29da6eed2dab8b, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.615132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.680208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:26.686153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1109/2223] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:32:26 2022] PUT /identity/v3/projects/b247e55a5f9e48cb88496964f6037ec7/users/135db966350b4e9ebb29da6eed2dab8b/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:32:26.697807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1e08b-9125-4f64-833b-5b6ae181f2f4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.698108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1e08b-9125-4f64-833b-5b6ae181f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.698393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1e08b-9125-4f64-833b-5b6ae181f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.726472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d8f1e08b-9125-4f64-833b-5b6ae181f2f4 None None] MFA Rules not processed for user `135db966350b4e9ebb29da6eed2dab8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:26.727852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d8f1e08b-9125-4f64-833b-5b6ae181f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:26.807259 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1115/2224] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:26.820806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:26.822613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:26.911383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f14a8-0138-42dc-a3f7-a82ab7adbfe8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:26.916053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:26.916898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:26.917184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:26.917510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:26.927801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:26.929313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9346e-c8bc-4a45-828d-dd954f86ed1c service glance] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:26.939634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1110/2225] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:32:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:32:34.797685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:34.798217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:34.798335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:34.800287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1116/2226] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:32:34.807116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd3fd-f21a-46fc-bf69-28547960648e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:34.807116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd3fd-f21a-46fc-bf69-28547960648e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:34.807116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccd3fd-f21a-46fc-bf69-28547960648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:34.866469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fcccd3fd-f21a-46fc-bf69-28547960648e None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:34.870499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fcccd3fd-f21a-46fc-bf69-28547960648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:34.949701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1111/2227] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:32:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:34.962503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:34.964216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:34.977837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc94dd6-5b0a-4434-b228-a02846411622 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:34.981684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:34.982310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:34.982592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:34.982866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:34.994036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:34.995501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945cc8d-4b68-407a-a27f-a5a5e9dfaf56 service glance] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:35.007251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1117/2228] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:32:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:32:35.279262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.279547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.279823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] PATH_INFO: `/` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.281277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1112/2229] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:32:35.286711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a443-5fc8-4791-bd84-e9c46cf04a5f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.287000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a443-5fc8-4791-bd84-e9c46cf04a5f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.287277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a443-5fc8-4791-bd84-e9c46cf04a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.306968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1cd3a443-5fc8-4791-bd84-e9c46cf04a5f None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:35.308203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1cd3a443-5fc8-4791-bd84-e9c46cf04a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:35.353202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:35.360824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1118/2230] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:32:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:35.372815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:35.374632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:35.388241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181ad6c-dbf7-4222-ba3c-1ee0324173b0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:35.391850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:35.392486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.392802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.393080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.400790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:35.402205 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95491f-f0c7-47d8-a4f4-88d532ecd47d service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:35.411742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1113/2231] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:32:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:32:35.493606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea512c-4e5e-4b07-b615-1b91be324126 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.493970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea512c-4e5e-4b07-b615-1b91be324126 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.494440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea512c-4e5e-4b07-b615-1b91be324126 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.496349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1119/2232] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:32:35.502444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d07e2-cb7b-4043-816c-e31b4a011ab6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.502732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d07e2-cb7b-4043-816c-e31b4a011ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.503036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d07e2-cb7b-4043-816c-e31b4a011ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.521587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8d1d07e2-cb7b-4043-816c-e31b4a011ab6 None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:35.522763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8d1d07e2-cb7b-4043-816c-e31b4a011ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:35.532409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1114/2233] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:32:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:35.542711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea512c-4e5e-4b07-b615-1b91be324126 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:35.554205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea512c-4e5e-4b07-b615-1b91be324126 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:35.558003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:35.558634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:35.558921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:35.559202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:35.570357 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:35.571863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:35.583627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1120/2234] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:32:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:32:39.451305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323194a-177a-4230-a8a1-c2691114d542 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:39.451829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323194a-177a-4230-a8a1-c2691114d542 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:39.451956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323194a-177a-4230-a8a1-c2691114d542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:39.470866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a323194a-177a-4230-a8a1-c2691114d542 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:32:39.472062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a323194a-177a-4230-a8a1-c2691114d542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:39.490060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:39.528943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1115/2235] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:32:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:32:39.541669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:32:39.543225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:32:39.610317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28864d1-addc-4d05-8ad1-d1314bccce9e service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:32:39.615397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:32:39.616034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:32:39.616346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:32:39.616653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:32:39.624535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:32:39.626131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:32:39.634583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1121/2236] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:32:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:32:49.523031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=15 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:32:49.631542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=39 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:17.525001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:17.525513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:17.525611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:17.543572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:17.544835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:17.551917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1116/2237] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:33:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:17.574351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:17.585081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3887b34-fd49-49b1-ad32-7edf821b2d1f service swift] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:17.590249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:17.590876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:17.591155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:17.591455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:17.599773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:17.601990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb001e9-b284-4e97-9d91-8c3d9c93c61a service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:17.610502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1122/2238] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:33:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:33:20.541136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:20.606877 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffdf0e-0f3d-4a39-8894-211e8d1afc13 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:20.611332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:20.612846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:20.612846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:20.612846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] PATH_INFO: `/v3/users/135db966350b4e9ebb29da6eed2dab8b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:20.614126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135db966350b4e9ebb29da6eed2dab8b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:20.616042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:20.793833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135db966350b4e9ebb29da6eed2dab8b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:20.801185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135db966350b4e9ebb29da6eed2dab8b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:20.922500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135db966350b4e9ebb29da6eed2dab8b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:20.923131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135db966350b4e9ebb29da6eed2dab8b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:20.923742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135db966350b4e9ebb29da6eed2dab8b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:20.941592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1117/2239] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:33:20 2022] DELETE /identity/v3/users/135db966350b4e9ebb29da6eed2dab8b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:33:20.946064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:20.946555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:20.946658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:20.964365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:20.965611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:21.026019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1123/2240] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:21.035560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:21.037009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:21.094901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe288c6-705b-4471-86b6-6485968ca475 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:21.098637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:21.099299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.099578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.099849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.109541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:21.111062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5af6c-1d51-4067-82be-4cd231662cd5 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:21.120772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1118/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:21.524219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:21.525717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:21.538887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fc775-aaf2-422c-a17a-a770acf99431 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:21.543786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:21.545467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.545785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.546046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] PATH_INFO: `/v3/projects/b247e55a5f9e48cb88496964f6037ec7` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.547371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b247e55a5f9e48cb88496964f6037ec7)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:21.549430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:21.669380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b247e55a5f9e48cb88496964f6037ec7', 'request_id': 'req-76133a18-fc71-4af3-9c9e-260dfa1cd713'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:21.674957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b247e55a5f9e48cb88496964f6037ec7', 'request_id': 'req-76133a18-fc71-4af3-9c9e-260dfa1cd713'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:21.681125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b247e55a5f9e48cb88496964f6037ec7', 'request_id': 'req-76133a18-fc71-4af3-9c9e-260dfa1cd713'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:21.753390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] The token cache is being invalidate because project b247e55a5f9e48cb88496964f6037ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:33:21.753615 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:21.754245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-76133a18-fc71-4af3-9c9e-260dfa1cd713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:33:21.755799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1124/2242] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:33:21 2022] DELETE /identity/v3/projects/b247e55a5f9e48cb88496964f6037ec7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:33:21.761138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.761609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.761718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.779911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:21.781129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:21.791248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:21.840568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1119/2243] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:21.848757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.849046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.849322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.868427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:21.869661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:21.877550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1125/2244] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:33:21.881574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:21.891979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23b4a2-5e96-4287-aa43-de5460820b7d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:21.896696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:21.897344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.897629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.897907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.899249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:33:21.899559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:21.901202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:21.912681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1120/2245] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:33:21.916691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:21.925890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719272f-85a6-4768-99e3-0de9f0b38384 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:21.930484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:21.931129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:21.931410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:21.931676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:21.932364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:21.934346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.044108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1126/2246] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:33:21 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:33:22.048445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:22.058870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c62c0e-2e0a-45bc-bd18-a1692ba21d82 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:22.115375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:22.116032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.116320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.116621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.117327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:22.119230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.213664 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1121/2247] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:33:22 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:33:22.217835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:22.229035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8b00d-e102-48f4-a8f1-aa925f39ef3e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:22.233974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:22.234625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.234923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.235183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.235917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:22.237607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.249627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1127/2248] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:33:22.253469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:22.263503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c653b-9513-4215-94d9-7879a6fed6b2 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:22.268004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:22.268672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.268958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.269234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.269981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:22.271576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d3873-2728-4503-960d-7f79729e9b76 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.282190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1122/2249] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:33:22.286102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:22.296628 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907832-6f68-4f96-bbec-fd1ac35de166 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:22.301197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:22.302055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.302336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.302611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] PATH_INFO: `/v3/projects/ae0e917f30d3430bb088ca3d0afe1a02/users/9db3cd25f4c94640854677709d10ccbe/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.316410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0e917f30d3430bb088ca3d0afe1a02, user_id=9db3cd25f4c94640854677709d10ccbe, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:22.319103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.376296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1128/2250] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:33:22 2022] PUT /identity/v3/projects/ae0e917f30d3430bb088ca3d0afe1a02/users/9db3cd25f4c94640854677709d10ccbe/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:33:22.381239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.381580 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.381860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.400053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:22.401275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:22.461243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1123/2251] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:33:22.470649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:22.472405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:22.482503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:22.529867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994557-9708-4b2f-b68b-e84cde357344 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:22.533537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:22.534175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:22.534460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:22.534737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:22.544429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:22.545933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:22.555790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1129/2252] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:24.314158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:24.316018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:24.345111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:24.376363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf858f5-7797-4502-b0b3-ab5c6f57581e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:24.380413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:24.381163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:24.381466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:24.381729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:24.384710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:24.472300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:24.474675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a44dc-5e51-4cd2-b12d-ed96434f9062 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:24.498216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1124/2253] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:27.307888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:27.317341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445155be-6949-4cf3-b47c-4bacefc4eeca service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:27.321536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:27.322167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:27.322446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:27.322717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:27.324507 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:27.338590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:27.340191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:27.352358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1130/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:28.759994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:28.760495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:28.760642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:28.789318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] MFA Rules not processed for user `9db3cd25f4c94640854677709d10ccbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:28.790442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:28.855091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1125/2255] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:28.859816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:28.871748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aca17-b558-45c8-9046-ed1000dbadc4 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:28.877696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:28.877696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:28.877696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:28.877970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:28.878549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:28.881167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:28.993963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1131/2256] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:33:28 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:33:28.999846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:29.012941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ae41-09af-4677-9a79-50142a56a016 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:29.077692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:29.092784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:29.093647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:29.094364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:29.095320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:29.096644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:29.105973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:29.198690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1126/2257] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:33:28 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:33:29.202532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:29.217764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d3020-30be-4eb1-b64f-7b6d805ae58f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:29.223350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:29.224115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:29.224401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:29.224715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:29.225593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:29.228132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:29.243589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1132/2258] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:33:29.247802 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:29.258247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f453-e2b3-487b-b9ec-f49846c59f02 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:29.262911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:29.263782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:29.264058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:29.264327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] PATH_INFO: `/v3/projects/66b4551152af4d9fad50672dec4ad3a1/users/cf564ae80a0a47ba93acb18832cd250d/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:29.279790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b4551152af4d9fad50672dec4ad3a1, user_id=cf564ae80a0a47ba93acb18832cd250d, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:29.282482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:29.369602 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1127/2259] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:33:29 2022] PUT /identity/v3/projects/66b4551152af4d9fad50672dec4ad3a1/users/cf564ae80a0a47ba93acb18832cd250d/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:33:29.379887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:29.385400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea521a-05c4-4e0b-8989-a69f19581f04 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:29.482855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:29.483575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:29.483858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:29.484134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:29.484908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:29.486712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:29.497841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1133/2260] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:33:29.501854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:29.518447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7dff-5b21-4ffe-bef4-1a2860da225d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:29.527778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:29.529004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:29.529652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:29.530146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] PATH_INFO: `/v3/projects/66b4551152af4d9fad50672dec4ad3a1/users/cf564ae80a0a47ba93acb18832cd250d/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:29.534249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b4551152af4d9fad50672dec4ad3a1, user_id=cf564ae80a0a47ba93acb18832cd250d, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:29.537807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d566-5efa-4c07-9f5c-e88bb43e4d9e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:29.550725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1128/2261] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:33:29 2022] PUT /identity/v3/projects/66b4551152af4d9fad50672dec4ad3a1/users/cf564ae80a0a47ba93acb18832cd250d/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:33:30.636448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:30.638212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:30.704267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:30.722511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddc8a2-47dc-489a-b0ef-6183ff1a54a4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:30.797788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:30.799048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:30.809323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:30.809765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:30.815292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:30.899283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:30.902018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:30.914380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1134/2262] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:35.421406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53753cc-0cee-4208-95d4-c5e4d45dfdce None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:35.421893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53753cc-0cee-4208-95d4-c5e4d45dfdce None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:35.422110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53753cc-0cee-4208-95d4-c5e4d45dfdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:35.449594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b53753cc-0cee-4208-95d4-c5e4d45dfdce None None] MFA Rules not processed for user `cf564ae80a0a47ba93acb18832cd250d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:35.450998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b53753cc-0cee-4208-95d4-c5e4d45dfdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:35.534799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1129/2263] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:35.547900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:35.559088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588606b2-e7c3-48bb-a746-d0ef8f0bc58a service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:35.564562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:35.565231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:35.565513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:35.565784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:35.630065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:35.631705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:35.641676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1135/2264] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:38.412697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:38.413747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:38.413747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:38.432639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] MFA Rules not processed for user `9db3cd25f4c94640854677709d10ccbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:38.433853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:38.441096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1130/2265] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:39.812772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:39.825203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56d7e-6aec-4f33-b946-b46dfc79ab54 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:39.843468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:39.844407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:39.845145 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:39.845981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:39.847969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:39.887497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:39.927475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:39.930187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:39.943215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1136/2266] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:41.689257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:41.750749 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84aaba3-69c9-493c-a652-920ffac4ea87 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:41.754349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:41.755076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:41.755364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:41.755641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:41.758412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:41.761381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:33:41.762769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:41.768918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:41.770359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fe383-8953-4fdf-8b67-7e19fe5cc0ab service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:41.777772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beead7-f028-4ae5-a7e1-0b2466bc1363 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:33:41.779389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1131/2267] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:33:41.782520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:33:41.783150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:41.783434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:41.783712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:41.795644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:33:41.797225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:33:41.806741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1137/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:33:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:33:44.847122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:33:44.847663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:33:44.847663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:33:44.866621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] MFA Rules not processed for user `75f6d2e28f024875bda0b20f751bdf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:33:44.868402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:33:44.875776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1132/2269] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:33:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:33:51.803361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=19 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:33:54.861066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:03.745564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:03.753980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc90c7-939f-402c-9398-430b54a065bd service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:03.757809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:03.758447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:03.758732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:03.759040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:03.760867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:03.770529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:03.771955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:03.780345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1138/2270] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:04.449331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:04.450787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:04.523978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a8588-f2ea-4351-a8b4-2584ddd9c28f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:04.529175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:04.530336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:04.530619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:04.530952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:04.532891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:04.544543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:04.546855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:04.556770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1133/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:04.875834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:04.893230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bd3fa-16c2-40e6-9aed-3615672470c6 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:04.898677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:04.899308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:04.899592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:04.899867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:04.901786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:04.960278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:04.961895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:04.972352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1139/2272] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:08.863030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:08.864575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:08.884172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83a10-63e6-4d5f-9ba1-426e45c724d8 service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:08.890862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:08.891670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:08.891967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:08.892244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:08.894350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:08.910312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:08.912477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:08.925611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1134/2273] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:10.745380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:10.760999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90c9e2-a8dc-4d7a-8ec8-b46d75774c76 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:10.776224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:10.777262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:10.777799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:10.778297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:10.780384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:10.794481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:10.796286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ee25-46e1-4782-8353-4e672091d69e service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:10.809959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1140/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:18.919765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=38 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:20.803577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=40 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:22.777254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:22.788455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578d2a9-79a3-4647-9353-39605d4dd45b service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:22.794290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:22.794948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:22.795225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:22.795496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:22.797439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:22.811054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:22.812735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:22.824202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1135/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:23.739405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:23.739938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:23.740046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:23.757390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] MFA Rules not processed for user `2f32356bb0bd4234ba817db581b62aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:34:23.758605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:23.765853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1141/2276] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:34:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:34:26.077141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:26.087801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac03f0e-fd33-414e-bf36-aebb1e4a932c service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:26.091558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:26.092194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:26.092479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:26.092793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:26.102975 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:26.104459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f9e9f-a797-431c-868f-dce1a4654f89 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:26.114359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1136/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:26.577593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:26.595772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa436-f988-4d61-a115-8ad95df3f7b2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:26.618957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:26.620247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:26.622138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:26.627277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:26.628653 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:26.718109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:26.721095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:26.735996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1142/2278] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:36.110024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=12 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:36.729172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=31 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:41.437624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45162f-1a5e-41fc-8ee3-b21e0c73ba8e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:41.438183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45162f-1a5e-41fc-8ee3-b21e0c73ba8e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:41.438183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45162f-1a5e-41fc-8ee3-b21e0c73ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:41.456914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-df45162f-1a5e-41fc-8ee3-b21e0c73ba8e None None] MFA Rules not processed for user `06e30bbe21af428faa5ec0eab414abf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:34:41.464683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-df45162f-1a5e-41fc-8ee3-b21e0c73ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:41.465111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1137/2279] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:34:41.767841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:34:41.777225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21757d6-b1f3-4429-b165-1943e67f5a3c service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:34:41.781673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:34:41.782311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:41.782595 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:41.782872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:41.784730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:41.794738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:34:41.796184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:34:41.804841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1143/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:34:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:34:51.451309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:51.800729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:34:59.943844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:34:59.944350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:34:59.944466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:34:59.962507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] MFA Rules not processed for user `0346f4e3855142059983083e71c2c0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:34:59.963735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:34:59.970638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1138/2281] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:34:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:09.956599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:23.151699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:23.161504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bcbc7-72e5-4a42-a2a0-8181b24656da service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:23.165284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:23.165909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:23.166195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:23.166468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:23.168317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:23.246244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:23.247665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:23.257729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1144/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:27.019577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:27.022442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:27.044041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21836289-32c2-4a1b-90c5-0238d4bfab65 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:27.048652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:27.049531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:27.049812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:27.050085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] PATH_INFO: `/v3/users/75f6d2e28f024875bda0b20f751bdf0d` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:27.051268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f6d2e28f024875bda0b20f751bdf0d)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:27.053268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:27.322155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f6d2e28f024875bda0b20f751bdf0d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.329241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f6d2e28f024875bda0b20f751bdf0d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.428945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f6d2e28f024875bda0b20f751bdf0d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.433501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f6d2e28f024875bda0b20f751bdf0d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.433501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ce17f819-df1a-4250-8815-bc750ee76bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f6d2e28f024875bda0b20f751bdf0d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.449510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1139/2283] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:35:27 2022] DELETE /identity/v3/users/75f6d2e28f024875bda0b20f751bdf0d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:27.724109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:27.733291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8dcf-76b9-4dd8-9db3-3a0e543309d9 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:27.798625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:27.799481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:27.799762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:27.800039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] PATH_INFO: `/v3/projects/856d592872fa47a2b628f1c3e482b2fd` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:27.801235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=856d592872fa47a2b628f1c3e482b2fd)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:27.803792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:27.919680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856d592872fa47a2b628f1c3e482b2fd', 'request_id': 'req-043f1c09-df11-4e76-9aec-f8d2c4942a66'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.925724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856d592872fa47a2b628f1c3e482b2fd', 'request_id': 'req-043f1c09-df11-4e76-9aec-f8d2c4942a66'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:27.932268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856d592872fa47a2b628f1c3e482b2fd', 'request_id': 'req-043f1c09-df11-4e76-9aec-f8d2c4942a66'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:28.011655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] The token cache is being invalidate because project 856d592872fa47a2b628f1c3e482b2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:35:28.011944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:28.012568 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-043f1c09-df11-4e76-9aec-f8d2c4942a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:28.014182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1145/2284] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:35:27 2022] DELETE /identity/v3/projects/856d592872fa47a2b628f1c3e482b2fd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:28.019261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.019556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.019867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.037442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:28.038643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:28.097422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1140/2285] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:35:28.105315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.105606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.105955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.124918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:28.126282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:28.134861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1146/2286] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:28.138902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.148820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df897d0f-330e-474e-8917-0f2af39383c6 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.153385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.154035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.154321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.154599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.155325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:35:28.155638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.157281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.169008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1141/2287] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:35:28.173301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.183041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e862a-fc66-4f63-85cd-19ae73de4b29 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.187740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.188379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.188709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.188986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.189678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.191617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.205630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:28.298425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1147/2288] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:28 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:35:28.303554 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.313476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82295a9-b28c-4fb9-ad3d-b5d8c3922d8e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.369514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.370168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.370451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.370727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.371425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.373387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.380331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:28.492426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1142/2289] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:28 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:28.496569 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.506897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219a76b-efb4-4e5e-81c4-013dbdfcf477 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.511482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.512132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.512414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.512719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.513530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.515091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.525238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1148/2290] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:28.529033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.539860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7461d-8f50-4d40-b03b-b69346e8c8b1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.550591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.551315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.552620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.552620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.553288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.554881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7588-b37a-4ca6-8209-3f32c2b1ba93 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.566332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1143/2291] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:35:28.570571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.580701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa7878-90cf-4423-ba86-719fe4ca2c9d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.585455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.586319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.586605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.586880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] PATH_INFO: `/v3/projects/2de6f435f741481d8154c8d68ae6d1ef/users/8162318f4542487a953905be69d40eac/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.601925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de6f435f741481d8154c8d68ae6d1ef, user_id=8162318f4542487a953905be69d40eac, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.604940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.680509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1149/2292] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:28 2022] PUT /identity/v3/projects/2de6f435f741481d8154c8d68ae6d1ef/users/8162318f4542487a953905be69d40eac/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:28.685485 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221cf59-c401-4c56-9300-3c162e6cc326 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.685769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221cf59-c401-4c56-9300-3c162e6cc326 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.686043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221cf59-c401-4c56-9300-3c162e6cc326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.702818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9221cf59-c401-4c56-9300-3c162e6cc326 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:28.704270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9221cf59-c401-4c56-9300-3c162e6cc326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:28.763729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1144/2293] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:28.773809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:28.775559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:28.830684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc337279-667c-4c29-823c-093883b4d664 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:28.834843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:28.835470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:28.835749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:28.836019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:28.844910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:28.846328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:28.856415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1150/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:30.115209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.115722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.115822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.142262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] MFA Rules not processed for user `8162318f4542487a953905be69d40eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:30.143489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:30.213497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1145/2295] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:30.218037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.228617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d55b-62e0-4506-9bd7-0bcd6a8b432a service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.233592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.234258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.234543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.234829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.235530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.237528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.338676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1151/2296] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:30 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:30.342712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.354675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37ff7f-d45b-435e-9dd6-e48cc697f498 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.413208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.413883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.414174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.414454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.415144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.417134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.521771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:30.529248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1146/2297] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:30 2022] POST /identity/v3/users => generated 308 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:30.531656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.542623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b6924-45f8-439d-a473-a291e145116c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.547350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.548044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.548353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.552785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.552785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.552785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.561459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1152/2298] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:30.565560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.575981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1907885-a877-47d9-a351-779390b09232 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.580694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.581564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.581849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.582126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] PATH_INFO: `/v3/projects/eaff7f7f9cef44b68b01252027465561/users/003d5932671e48d39aed1deb0cd38919/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.597150 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaff7f7f9cef44b68b01252027465561, user_id=003d5932671e48d39aed1deb0cd38919, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.599795 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.685910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1147/2299] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:30 2022] PUT /identity/v3/projects/eaff7f7f9cef44b68b01252027465561/users/003d5932671e48d39aed1deb0cd38919/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:30.689832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.700560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9438c-0c51-4431-9aac-a9af9f231093 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.758651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.759324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.759611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.759891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.760690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.762265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df925744-a0b4-48e2-b5de-0b4f8e2d6f05 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.772850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1153/2300] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:30.776974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:30.787742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ad4d-d31c-4cb8-afae-e1aa8ad06e60 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:30.792424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:30.793376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:30.793611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:30.793884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] PATH_INFO: `/v3/projects/eaff7f7f9cef44b68b01252027465561/users/003d5932671e48d39aed1deb0cd38919/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:30.795831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaff7f7f9cef44b68b01252027465561, user_id=003d5932671e48d39aed1deb0cd38919, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:30.798493 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:30.808889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1148/2301] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:30 2022] PUT /identity/v3/projects/eaff7f7f9cef44b68b01252027465561/users/003d5932671e48d39aed1deb0cd38919/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:32.383879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:32.384340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:32.384630 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:32.412153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] MFA Rules not processed for user `003d5932671e48d39aed1deb0cd38919`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:32.413594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:32.419025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:32.483223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1154/2302] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:32 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:33.002319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:33.005088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:33.077662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21549207-c003-4eb1-b574-11913a964243 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:33.082615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:33.083255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:33.083538 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:33.083812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:33.085645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:33.096933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:33.098998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:33.109010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1149/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:33.799085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:33.862371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88718-e99e-4f78-8441-6cad98dbeb17 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:33.866936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:33.867820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:33.868101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:33.868376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] PATH_INFO: `/v3/users/8162318f4542487a953905be69d40eac` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:33.869620 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8162318f4542487a953905be69d40eac)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:33.871546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:34.088233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8162318f4542487a953905be69d40eac'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.095799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8162318f4542487a953905be69d40eac'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.096711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8162318f4542487a953905be69d40eac'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.194548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8162318f4542487a953905be69d40eac'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.195159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8162318f4542487a953905be69d40eac'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.213535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1155/2304] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:35:33 2022] DELETE /identity/v3/users/8162318f4542487a953905be69d40eac => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:34.453078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:34.464557 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ba5f2-1c64-4489-bfc2-3b36272c140f service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:34.524189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:34.525172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:34.525462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:34.525740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] PATH_INFO: `/v3/projects/2de6f435f741481d8154c8d68ae6d1ef` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:34.526929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de6f435f741481d8154c8d68ae6d1ef)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:34.528913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:34.664714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de6f435f741481d8154c8d68ae6d1ef', 'request_id': 'req-70634d2c-9564-4bde-90e7-d641b12b5a1e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.666578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:34.671079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de6f435f741481d8154c8d68ae6d1ef', 'request_id': 'req-70634d2c-9564-4bde-90e7-d641b12b5a1e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.676635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de6f435f741481d8154c8d68ae6d1ef', 'request_id': 'req-70634d2c-9564-4bde-90e7-d641b12b5a1e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.759312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] The token cache is being invalidate because project 2de6f435f741481d8154c8d68ae6d1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:35:34.759531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.760178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:34.761744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1150/2305] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:35:34 2022] DELETE /identity/v3/projects/2de6f435f741481d8154c8d68ae6d1ef => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:34.766043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:34.767473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:34.780840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0cc9-1a7f-432e-baea-5f347346fb2c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:34.799452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:34.836690 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:34.839122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:34.839122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:34.839122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] PATH_INFO: `/v3/users/003d5932671e48d39aed1deb0cd38919` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:34.839780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003d5932671e48d39aed1deb0cd38919)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:34.841729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:35.082913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003d5932671e48d39aed1deb0cd38919'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.091625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003d5932671e48d39aed1deb0cd38919'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.092287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003d5932671e48d39aed1deb0cd38919'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.194186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003d5932671e48d39aed1deb0cd38919'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.194854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-989fe977-754b-4066-a323-7e7fbfa08db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003d5932671e48d39aed1deb0cd38919'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.212879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1156/2306] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:35:34 2022] DELETE /identity/v3/users/003d5932671e48d39aed1deb0cd38919 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:35.443777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:35.445895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:35.460883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634d2c-9564-4bde-90e7-d641b12b5a1e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:35.521309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:35.522157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:35.522435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:35.522705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] PATH_INFO: `/v3/projects/eaff7f7f9cef44b68b01252027465561` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:35.523892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaff7f7f9cef44b68b01252027465561)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:35.525884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:35.623277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaff7f7f9cef44b68b01252027465561', 'request_id': 'req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.629857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaff7f7f9cef44b68b01252027465561', 'request_id': 'req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.636179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaff7f7f9cef44b68b01252027465561', 'request_id': 'req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.702077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] The token cache is being invalidate because project eaff7f7f9cef44b68b01252027465561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:35:35.702287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.702912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c9ee16d4-d44f-4676-bfb0-a4f9524ed359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:35.704453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1151/2307] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:35:35 2022] DELETE /identity/v3/projects/eaff7f7f9cef44b68b01252027465561 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:35.709341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:35.709651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:35.710404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:35.728217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:35.729709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:35.789732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1157/2308] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:35.797442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:35.797728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:35.798002 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:35.815692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:35.816915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:35.825154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1152/2309] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:35:35.829229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:35.839444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25d7d0-d421-4ca4-b8ef-71e35dfd28c9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:35.843985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:35.844655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:35.844941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:35.845212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:35.845927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:35:35.846225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:35.847805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:35.859371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1158/2310] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:35:35.863419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:35.872404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94701-45b7-44de-9d40-382d7705afb3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:35.877116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:35.877748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:35.878026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:35.878290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:35.878978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:35.880920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:35.990148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1153/2311] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:35 2022] POST /identity/v3/projects => generated 407 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:35.996764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.007532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8988-f8b3-47d5-bad7-b95b06aa72b3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.033754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:36.062440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.063100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.063382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.063662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.064343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.066284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.171712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1159/2312] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:35 2022] POST /identity/v3/users => generated 304 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:36.175979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.186775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0064e95-8c38-40cc-806a-4a0a44590c41 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.191440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.192086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.192371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.192680 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.193475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.195738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.205959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1154/2313] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:36.218692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.228497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c70f4c-c411-44c1-ab84-69e59c826cd4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.233404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.234261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.234718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.235381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.236159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.237891 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1f863-5aa4-4211-a093-417c7a203687 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.248909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1160/2314] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:35:36.252770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.263364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9a35-543a-4e79-bed2-02e862666671 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.268099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.269046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.269325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.269596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] PATH_INFO: `/v3/projects/1c524ff387e84f54830b3d57b4f9b62e/users/2d16024b2689495dab7bff8f7670bcf5/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.283909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c524ff387e84f54830b3d57b4f9b62e, user_id=2d16024b2689495dab7bff8f7670bcf5, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.286575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.388115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1155/2315] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:36 2022] PUT /identity/v3/projects/1c524ff387e84f54830b3d57b4f9b62e/users/2d16024b2689495dab7bff8f7670bcf5/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:36.398700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.401429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.401429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.426056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] MFA Rules not processed for user `2d16024b2689495dab7bff8f7670bcf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:36.427311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:36.490829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1161/2316] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:36.495088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.504274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:36.556915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d08be-324f-4d24-8217-d40a53dae19e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.561583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.562258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.562535 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.562806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.563494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.565407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.679463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1156/2317] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:36 2022] POST /identity/v3/projects => generated 407 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:36.683556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.694952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fa227-9145-4a86-823d-d4970e2097e6 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.750554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.751234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.751518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.751792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.752468 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.754467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.844438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1162/2318] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:36 2022] POST /identity/v3/users => generated 310 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:36.848541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.859808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb53d9-738e-4660-adfd-0e20744d9339 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.864596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.865258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.865549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.866390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.867117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.868737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.879326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1157/2319] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:36.884183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:36.894213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79228de-cfdf-48ce-a7e0-ffaac9c24e0e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:36.898905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:36.899775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:36.900056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:36.900329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] PATH_INFO: `/v3/projects/a83e7609a854444b9c1fa7e7c529122f/users/24dda0b226e445df88e16457957260ed/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:36.914881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83e7609a854444b9c1fa7e7c529122f, user_id=24dda0b226e445df88e16457957260ed, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:36.917590 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:36.994814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1163/2320] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:36 2022] PUT /identity/v3/projects/a83e7609a854444b9c1fa7e7c529122f/users/24dda0b226e445df88e16457957260ed/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:35:36.998738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:37.010848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bad191-3558-4264-9cc9-686135bd05e8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:37.068605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:37.069921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:37.070203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:37.070477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:37.071195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:37.072799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d05df-2a82-4f9d-af08-7f50912eaf98 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:37.082481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1158/2321] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:37.086837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:37.092601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:37.097867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22179ca-0ba4-4f73-82be-83dadbc186d0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:37.102737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:37.103892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:37.104530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:37.104852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] PATH_INFO: `/v3/projects/a83e7609a854444b9c1fa7e7c529122f/users/24dda0b226e445df88e16457957260ed/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:37.107151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83e7609a854444b9c1fa7e7c529122f, user_id=24dda0b226e445df88e16457957260ed, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:37.109948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:37.120731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1164/2322] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:37 2022] PUT /identity/v3/projects/a83e7609a854444b9c1fa7e7c529122f/users/24dda0b226e445df88e16457957260ed/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:37.130780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:37.131124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:37.131944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:37.157551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] MFA Rules not processed for user `24dda0b226e445df88e16457957260ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:37.158750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:37.225779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1159/2323] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:37.236925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:37.238645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:37.296679 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f00d-1c58-4627-b943-32970d640e51 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:37.300257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:37.300928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:37.301216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:37.301496 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:37.310459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:37.311890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:37.321806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1165/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:37 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:37.694590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:37.699340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:37.704767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90768bee-160c-41ca-8b44-87fa6767c9b9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:37.714592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:37.714592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:37.714592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:37.714592 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:37.772876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:37.774312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c1920-cf1e-4dba-ad74-bd63d2556a9c service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:37.783540 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1160/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:47.317238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:47.779303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=27 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:49.824326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:49.886550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db511-1720-4250-9864-2a4096ef883d service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:49.891223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:49.892086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:49.892363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:49.892654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] PATH_INFO: `/v3/users/2d16024b2689495dab7bff8f7670bcf5` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:49.893858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d16024b2689495dab7bff8f7670bcf5)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:49.895781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:50.054733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d16024b2689495dab7bff8f7670bcf5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.061898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d16024b2689495dab7bff8f7670bcf5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.062561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d16024b2689495dab7bff8f7670bcf5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.137086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d16024b2689495dab7bff8f7670bcf5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.137650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d16024b2689495dab7bff8f7670bcf5'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.155592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1166/2326] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:35:49 2022] DELETE /identity/v3/users/2d16024b2689495dab7bff8f7670bcf5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:50.160098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:50.160564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:50.164525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:50.177902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:50.179755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:50.236956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1161/2327] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:50.247300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:50.248747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:50.306103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd89f-e7be-4656-b674-abcc7b4d530f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:50.309748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:50.310385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:50.310668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:50.310944 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:50.319861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:50.321320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:50.330693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1167/2328] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:35:50.755687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:50.757686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:50.769581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83bfa9-dfeb-4a4c-a300-754bfc4997b2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:50.774875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:50.775746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:50.776015 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:50.776289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] PATH_INFO: `/v3/projects/1c524ff387e84f54830b3d57b4f9b62e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:50.777518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c524ff387e84f54830b3d57b4f9b62e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:50.779658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:50.885501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c524ff387e84f54830b3d57b4f9b62e', 'request_id': 'req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.891659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c524ff387e84f54830b3d57b4f9b62e', 'request_id': 'req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.897213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c524ff387e84f54830b3d57b4f9b62e', 'request_id': 'req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.987775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] The token cache is being invalidate because project 1c524ff387e84f54830b3d57b4f9b62e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:35:50.988073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.988721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:50.990334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1162/2329] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:35:50 2022] DELETE /identity/v3/projects/1c524ff387e84f54830b3d57b4f9b62e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:50.994370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:51.003955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8a40a-b56f-4b19-829f-b8701d9923de service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:51.022374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:51.061459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:51.062324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:51.062621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:51.062884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] PATH_INFO: `/v3/users/24dda0b226e445df88e16457957260ed` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:51.064061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24dda0b226e445df88e16457957260ed)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:51.066003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:51.253521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24dda0b226e445df88e16457957260ed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.261004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24dda0b226e445df88e16457957260ed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.261604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24dda0b226e445df88e16457957260ed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.320546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24dda0b226e445df88e16457957260ed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.321206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-cbc5e2a0-e507-42b4-ab9c-fecf67538578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24dda0b226e445df88e16457957260ed'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.339413 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1168/2330] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:35:50 2022] DELETE /identity/v3/users/24dda0b226e445df88e16457957260ed => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:51.759834 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:51.761374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:51.776166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bf49-4b82-4586-90d5-cdc56a6fd65a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:51.833531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:51.834378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:51.834656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:51.834930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] PATH_INFO: `/v3/projects/a83e7609a854444b9c1fa7e7c529122f` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:51.836696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83e7609a854444b9c1fa7e7c529122f)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:51.838573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:51.932769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:51.948920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83e7609a854444b9c1fa7e7c529122f', 'request_id': 'req-efd49328-d71e-499c-aba4-acf6a86a7402'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.954865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83e7609a854444b9c1fa7e7c529122f', 'request_id': 'req-efd49328-d71e-499c-aba4-acf6a86a7402'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:51.960156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83e7609a854444b9c1fa7e7c529122f', 'request_id': 'req-efd49328-d71e-499c-aba4-acf6a86a7402'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:52.045362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] The token cache is being invalidate because project a83e7609a854444b9c1fa7e7c529122f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:35:52.045605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:52.046230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-efd49328-d71e-499c-aba4-acf6a86a7402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:35:52.047837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1163/2331] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:35:51 2022] DELETE /identity/v3/projects/a83e7609a854444b9c1fa7e7c529122f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:52.053511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.056660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.056660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.072760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:52.074043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:52.134466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1169/2332] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:52.143293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.143585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.143892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.162055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:52.163255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:52.171580 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1164/2333] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:52.175864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.186208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb3de6-59cb-4291-a22e-aeed06ef421c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.190863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.191538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.191809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.192084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.192840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:35:52.193179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.195423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.207095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1170/2334] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:35:52.211271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.220246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0bee9-6640-4b9d-ba2a-7ef81429f2b8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.224817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.225448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.226230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.226498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.227209 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.229368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.321171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1165/2335] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:52.325197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.335474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3e66e-0a51-4bba-8d16-9dc4dc412b76 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.362197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:52.392608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.393401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.393802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.394085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.394835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.396778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.530975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1171/2336] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:52.535044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.545988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368d4c-91f0-46f0-a6da-71b939b87e50 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.550480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.551726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.551726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.551726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.552414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.554735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.565255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1166/2337] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:52.569238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.579497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a2591-c413-4be8-b02a-8d760e63f6ff admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.584206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.585470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.585760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.586038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.586889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.588538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d791e92-9944-4d8a-8a46-ad37adce000f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.598439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1172/2338] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:52.603054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.613062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adb0b0-f264-489a-ba35-a35c9fa558ed admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.617479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.618345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.618623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.618893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] PATH_INFO: `/v3/projects/f0b4648705aa4d5aa39a5df8f709c6f1/users/9d60bdea08f342658b706b44374bef08/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.633032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b4648705aa4d5aa39a5df8f709c6f1, user_id=9d60bdea08f342658b706b44374bef08, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.635648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.708312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1167/2339] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:52 2022] PUT /identity/v3/projects/f0b4648705aa4d5aa39a5df8f709c6f1/users/9d60bdea08f342658b706b44374bef08/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:52.719050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.719339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.719614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.747333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] MFA Rules not processed for user `9d60bdea08f342658b706b44374bef08`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:52.748533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:52.811172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1173/2340] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:52.815572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.878590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1306e6d-401e-4936-9b2c-8b1ae5832533 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:52.883996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:52.884716 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:52.885000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:52.885271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:52.885990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:52.887946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:52.978912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1168/2341] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:52.983166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:52.997081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef3bc2-889c-4a4a-a741-cd2f1c9d033c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.068298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.069024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.069267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.069536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.070206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.072113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.222685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1174/2342] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:35:52 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:35:53.227191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:53.238034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bf1-25b7-444a-b036-ad524b93d1c3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.243234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.243884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.244167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.244468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.245232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.246811 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.257208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1169/2343] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:53.261106 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:53.270910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111ebe7-7a9a-4e7d-b168-e7a8d9505ead admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.275435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.276298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.276602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.276884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] PATH_INFO: `/v3/projects/4cf35e7b16f044238e1a863993edf72f/users/1d1cd552fd1744f5926aad60949640c2/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.291657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf35e7b16f044238e1a863993edf72f, user_id=1d1cd552fd1744f5926aad60949640c2, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.294342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.368859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1175/2344] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:53 2022] PUT /identity/v3/projects/4cf35e7b16f044238e1a863993edf72f/users/1d1cd552fd1744f5926aad60949640c2/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:53.372790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:53.382626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55d582-abc1-4bf6-87c2-cf6c50214c40 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.402497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:53.439009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.439667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.439953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.440229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.440978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.442555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6461b8-8f41-4072-bf0f-6e4faa0bfd57 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.452353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1170/2345] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:35:53.456390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:53.462378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:35:53.467756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defff62-725e-4984-828a-548a2c508616 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.472347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.478588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.478588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.478588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] PATH_INFO: `/v3/projects/4cf35e7b16f044238e1a863993edf72f/users/1d1cd552fd1744f5926aad60949640c2/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.478588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf35e7b16f044238e1a863993edf72f, user_id=1d1cd552fd1744f5926aad60949640c2, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.478932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.488636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1176/2346] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:35:53 2022] PUT /identity/v3/projects/4cf35e7b16f044238e1a863993edf72f/users/1d1cd552fd1744f5926aad60949640c2/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:35:53.499053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.499349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.499658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.525878 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] MFA Rules not processed for user `1d1cd552fd1744f5926aad60949640c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:35:53.527060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:53.594401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1171/2347] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:35:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:35:53.605258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:35:53.607002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:35:53.665920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56e01-b760-496e-aef2-5722fa2336dd admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:35:53.669611 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:35:53.670247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:35:53.670529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:35:53.670805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:35:53.726066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:35:53.727490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:35:53.736552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1177/2348] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:35:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:02.210151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:02.219585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239c772-bf62-42c6-ab4c-5677c00ed1e5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:02.223136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:02.223764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:02.224050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:02.224323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:02.233708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:02.235120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ad365-eea8-4d82-9da8-29618c3c079b service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:02.245035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1172/2349] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:03.732561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:09.076973 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:09.139107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc57d1-19ed-43c3-8fe6-a43a523527ad service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:09.143608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:09.144485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:09.144790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:09.145078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] PATH_INFO: `/v3/users/9d60bdea08f342658b706b44374bef08` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:09.146230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d60bdea08f342658b706b44374bef08)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:09.148153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:09.355672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d60bdea08f342658b706b44374bef08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:09.363960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d60bdea08f342658b706b44374bef08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:09.364643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d60bdea08f342658b706b44374bef08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:09.438232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d60bdea08f342658b706b44374bef08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:09.439061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d60bdea08f342658b706b44374bef08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:09.458355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1178/2350] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:36:09 2022] DELETE /identity/v3/users/9d60bdea08f342658b706b44374bef08 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:09.463043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:09.463539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:09.463642 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:09.481914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:09.483115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:09.542296 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1173/2351] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:09.552044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:09.553505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:09.613084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55735e8e-20ff-450b-9305-aa6ab5f3f69e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:09.616761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:09.617406 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:09.617689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:09.617964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:09.627456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:09.637608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:09.639200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1179/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:10.110532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:10.112087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:10.128869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764b746-f6a0-4349-a6dd-71e7bee021c1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:10.129700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:10.130606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:10.130951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:10.131108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] PATH_INFO: `/v3/projects/f0b4648705aa4d5aa39a5df8f709c6f1` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:10.132293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b4648705aa4d5aa39a5df8f709c6f1)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:10.134236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:10.237813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:10.252213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b4648705aa4d5aa39a5df8f709c6f1', 'request_id': 'req-354280c1-764d-4a39-9cef-c100383a4a37'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.258578 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b4648705aa4d5aa39a5df8f709c6f1', 'request_id': 'req-354280c1-764d-4a39-9cef-c100383a4a37'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.264523 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b4648705aa4d5aa39a5df8f709c6f1', 'request_id': 'req-354280c1-764d-4a39-9cef-c100383a4a37'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.348609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] The token cache is being invalidate because project f0b4648705aa4d5aa39a5df8f709c6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:36:10.348904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.352141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.352141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1174/2353] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:36:10 2022] DELETE /identity/v3/projects/f0b4648705aa4d5aa39a5df8f709c6f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:10.355269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:10.366278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a52b-8be9-4ade-8cfc-8c71f2824668 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:10.385504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:10.422743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:10.423608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:10.423889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:10.424161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] PATH_INFO: `/v3/users/1d1cd552fd1744f5926aad60949640c2` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:10.425373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1cd552fd1744f5926aad60949640c2)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:10.427292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:10.635704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1cd552fd1744f5926aad60949640c2'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.642731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1cd552fd1744f5926aad60949640c2'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.643509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1cd552fd1744f5926aad60949640c2'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.742149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1cd552fd1744f5926aad60949640c2'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.742786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-900b8630-acfd-4b8e-aba4-86daf68fd0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1cd552fd1744f5926aad60949640c2'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:10.761290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1180/2354] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:36:10 2022] DELETE /identity/v3/users/1d1cd552fd1744f5926aad60949640c2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:11.178364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:11.179771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:11.193982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354280c1-764d-4a39-9cef-c100383a4a37 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:11.253435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:11.254239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.254524 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.254799 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] PATH_INFO: `/v3/projects/4cf35e7b16f044238e1a863993edf72f` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.256097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf35e7b16f044238e1a863993edf72f)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:11.258189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:11.394587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf35e7b16f044238e1a863993edf72f', 'request_id': 'req-bbdad7e1-a89c-4588-a35c-e53ebc18797d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:11.401117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf35e7b16f044238e1a863993edf72f', 'request_id': 'req-bbdad7e1-a89c-4588-a35c-e53ebc18797d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:11.406699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf35e7b16f044238e1a863993edf72f', 'request_id': 'req-bbdad7e1-a89c-4588-a35c-e53ebc18797d'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:11.524672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] The token cache is being invalidate because project 4cf35e7b16f044238e1a863993edf72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:36:11.524897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:11.525517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-bbdad7e1-a89c-4588-a35c-e53ebc18797d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:11.527054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1175/2355] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:36:11 2022] DELETE /identity/v3/projects/4cf35e7b16f044238e1a863993edf72f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:11.533080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.533556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.533654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.551374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:11.552591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:11.612100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1181/2356] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:11.620130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.620421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.620755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.639468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:11.640711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:11.648837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1176/2357] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:11.652909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:11.662546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d348159-bafb-4609-9df9-982d673e4cd9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:11.667155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:11.667775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.668055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.668866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.669581 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:36:11.669885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:11.671476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:11.683037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1182/2358] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:36:11.687908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:11.696928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3dbe6-0b9c-4b3c-98c8-fc604c42eb5a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:11.701605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:11.702239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.702519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.702791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.703464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:11.705381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:11.829876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1177/2359] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:36:11 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:36:11.833930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:11.844139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b91bf-ac41-4bdc-8e53-4f8be8c081d4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:11.869854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:11.899052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:11.899714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:11.899997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:11.900272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:11.900989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:11.903280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:12.023091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1183/2360] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:36:11 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:36:12.029186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:12.037830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6da6-ea4e-4b7d-a7ca-3eb68cd3d983 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:12.042394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:12.043044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:12.044016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:12.044294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:12.045148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:12.046741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:12.056956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1178/2361] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:36:12.060961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:12.071031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b739-e3a3-4768-83b9-0c7029c4f864 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:12.075621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:12.076261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:12.076608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:12.076905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:12.077626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:12.079215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f39c05-80d1-4c17-ac53-dfa64ffd4689 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:12.089390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1184/2362] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:36:12.093729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:12.104115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601854b3-8766-42f4-9527-174d7bddd53c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:12.109459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:12.110323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:12.110604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:12.110882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] PATH_INFO: `/v3/projects/121cfa83e948429685eb5d0880277c7f/users/a43625dc12cf4c48baec38751b479840/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:12.113130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:12.125457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=121cfa83e948429685eb5d0880277c7f, user_id=a43625dc12cf4c48baec38751b479840, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:12.128078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:12.226468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1179/2363] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:36:12 2022] PUT /identity/v3/projects/121cfa83e948429685eb5d0880277c7f/users/a43625dc12cf4c48baec38751b479840/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:12.238376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:12.238666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:12.238995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:12.266846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] MFA Rules not processed for user `a43625dc12cf4c48baec38751b479840`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:12.268070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:12.334990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1185/2364] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:12.350707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:12.352411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:12.412801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7926e-50f9-4932-892b-a5a1d1f6ec9f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:12.416503 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:12.417174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:12.417458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:12.417730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:12.426494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:12.427699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a467b-5b34-4781-87b4-50d84f19c412 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:12.436909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1180/2365] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:22.331965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:22.432650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:24.618560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:24.681140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486a1e8-f0cd-4a9f-bc7c-a917e8a6b9a5 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:24.686381 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:24.687126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:24.687411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:24.687688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] PATH_INFO: `/v3/users/a43625dc12cf4c48baec38751b479840` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:24.688958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43625dc12cf4c48baec38751b479840)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:24.691078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:24.853705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43625dc12cf4c48baec38751b479840'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:24.861602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43625dc12cf4c48baec38751b479840'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:24.863456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43625dc12cf4c48baec38751b479840'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.090928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43625dc12cf4c48baec38751b479840'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.091487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43625dc12cf4c48baec38751b479840'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.110107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1186/2366] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:36:24 2022] DELETE /identity/v3/users/a43625dc12cf4c48baec38751b479840 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:36:25.114712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:25.115546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:25.115724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:25.134097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:25.135347 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:25.194481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1181/2367] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:25.204119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:25.205598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:25.265246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8df6d-0b5c-4429-85e9-20d1abb8b12c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:25.268968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:25.269614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:25.269901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:25.270178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:25.279901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:25.281402 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a262f9f-d8d8-48c1-a68a-c5f7ab1ee727 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:25.291397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1187/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:25.726494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:25.730438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:25.743270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730948f-db82-47a5-a0cd-ea1c122090f0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:25.749462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:25.750456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:25.750819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:25.751124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] PATH_INFO: `/v3/projects/121cfa83e948429685eb5d0880277c7f` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:25.752381 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121cfa83e948429685eb5d0880277c7f)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:25.754372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:25.868688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121cfa83e948429685eb5d0880277c7f', 'request_id': 'req-c2799109-8236-4032-92a8-cb0b265a58d8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.875172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121cfa83e948429685eb5d0880277c7f', 'request_id': 'req-c2799109-8236-4032-92a8-cb0b265a58d8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.881424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121cfa83e948429685eb5d0880277c7f', 'request_id': 'req-c2799109-8236-4032-92a8-cb0b265a58d8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.988715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] The token cache is being invalidate because project 121cfa83e948429685eb5d0880277c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:36:25.989300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.990123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c2799109-8236-4032-92a8-cb0b265a58d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:36:25.992004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1182/2369] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:36:25 2022] DELETE /identity/v3/projects/121cfa83e948429685eb5d0880277c7f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:25.997105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:25.997688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:25.997688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.017584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:26.019228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:26.042210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:26.095626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1188/2370] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:26.104478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.104884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.105237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.125846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:26.126979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:26.135079 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1183/2371] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:36:26.145844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.155878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201eb5c3-79c8-4b0d-9840-61f39d40c4f3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.160535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.165318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.177451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1189/2372] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:36:26.181822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.191388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b05b69-28b5-4c1e-8038-3a93f17922a9 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.196482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.197168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.197453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.197733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.198628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.200672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.281677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1184/2373] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:36:26 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:36:26.285559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.296449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6218-efad-4c82-8706-cbb7e794dac4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.351423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.352086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.352373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.352680 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.353642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.355564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.433551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1190/2374] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:36:26 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:36:26.437783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.448142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35abe61-e6ae-4f63-9649-b98dbd07650c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.452899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.453549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.453843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.454122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.454876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.456462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.467179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1185/2375] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:36:26.470783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.480990 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c522-127b-4c29-a45f-cd79a4948a70 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.487165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.487820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.488103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.488407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.489177 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.490780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d677b-287d-4677-81f3-d5408841b2a7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.500963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1191/2376] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:36:26.505303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.515996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab570890-2708-4ec3-813e-43d1248e0b18 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.520816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.522055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.522338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.522616 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] PATH_INFO: `/v3/projects/97e6c3c9845b44c29d506b8f4868d829/users/dc22b89a381844259fd11efea3c62b08/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.538487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e6c3c9845b44c29d506b8f4868d829, user_id=dc22b89a381844259fd11efea3c62b08, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.541187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.621974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1186/2377] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:36:26 2022] PUT /identity/v3/projects/97e6c3c9845b44c29d506b8f4868d829/users/dc22b89a381844259fd11efea3c62b08/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:36:26.632824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.633113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.633398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.659370 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] MFA Rules not processed for user `dc22b89a381844259fd11efea3c62b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:26.660604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:26.724949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1192/2378] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:26.736920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:26.738659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:26.760981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:26.802822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b59b-1917-4cd1-b1c4-12ebfcbe674f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:26.806892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:26.807541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:26.807833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:26.808111 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:26.818670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:26.820307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fb8f5-a2e9-4953-9e99-b5296ab6df98 service glance] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:26.831959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1187/2379] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:32.296303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:32.297814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:32.323815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:32.376320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568757b4-a6d2-4104-bd6b-fe96c07f27d3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:32.381382 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:32.382040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:32.382330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:32.382610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:32.384383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:32.398097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:32.399565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c40f-b279-40f7-b2fc-2e7dbb570982 service placement] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:32.410559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1193/2380] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:36:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:36:32.776957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:32.777544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:32.777544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:32.796418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] MFA Rules not processed for user `2f32356bb0bd4234ba817db581b62aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:32.797724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:32.854311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1188/2381] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:42.787818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=24 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:42.846809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=42 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:36:50.202706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:50.203218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:50.203311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:50.221126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:36:50.222327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:50.283634 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1194/2382] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:36:50.445064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:36:50.448740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:36:50.535830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794f8a6-5f7d-4296-a9e2-352e526572b8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:36:50.541478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:36:50.542139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:36:50.542432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:36:50.542711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:36:50.551781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:36:50.553466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756ed9-7d48-4837-bcff-f008631bf5df service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:36:50.564206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1189/2383] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:36:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 09:37:00.275921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=25 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:00.559299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=32 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:11.375455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:11.440360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3a7e4-cf7d-4c41-aef9-2e0f14ea1b96 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:11.444923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:11.445778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:11.446059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:11.446329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] PATH_INFO: `/v3/users/dc22b89a381844259fd11efea3c62b08` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:11.447509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc22b89a381844259fd11efea3c62b08)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:11.449455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:11.688619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc22b89a381844259fd11efea3c62b08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:11.696542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc22b89a381844259fd11efea3c62b08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:11.697735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc22b89a381844259fd11efea3c62b08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:11.821302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc22b89a381844259fd11efea3c62b08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:11.821928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc22b89a381844259fd11efea3c62b08'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:11.840354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1195/2384] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:37:11 2022] DELETE /identity/v3/users/dc22b89a381844259fd11efea3c62b08 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:37:11.849941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:11.850434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:11.850434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:11.870186 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:11.871395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:11.931157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1190/2385] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:11.941037 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:11.942524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:11.999105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940318c-86ea-42bb-b020-870d0c352e2c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:12.002748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:12.003389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:12.003704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:12.004010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:12.013729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:12.015237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91a883-f0b9-45ed-9791-7684537a26b9 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:12.025014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1196/2386] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:37:12.571542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:12.573010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:12.586546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f44ed9-5d8c-40fb-9e98-ff2bc2e33b0f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:12.592862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:12.593936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:12.594285 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:12.595117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] PATH_INFO: `/v3/projects/97e6c3c9845b44c29d506b8f4868d829` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:12.597358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e6c3c9845b44c29d506b8f4868d829)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:12.599617 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:12.787050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e6c3c9845b44c29d506b8f4868d829', 'request_id': 'req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:12.793358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e6c3c9845b44c29d506b8f4868d829', 'request_id': 'req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:12.799087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e6c3c9845b44c29d506b8f4868d829', 'request_id': 'req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:12.947280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] The token cache is being invalidate because project 97e6c3c9845b44c29d506b8f4868d829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:37:12.947947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:12.948442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cbf68e20-5c53-46b1-9c92-9f90ff5b1e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:12.950070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1191/2387] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:37:12 2022] DELETE /identity/v3/projects/97e6c3c9845b44c29d506b8f4868d829 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:37:12.955699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:12.956255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:12.956390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:12.985667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:12.986924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:13.016104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:13.075333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1197/2388] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:13.083872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.084418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.084919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.118618 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:13.122422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:13.134101 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1192/2389] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:13.139026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.149924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38f1ae-ed72-4f67-8d57-ebcf05c3ecc3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.156404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.157472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.157926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.158365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.159120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:37:13.159587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.161298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.179992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1198/2390] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:37:13.181164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.191416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8e422-d8d8-4885-9bfb-cb7ab956aacc None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.196940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.197936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.198368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.198776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.199569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.201537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.323693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1193/2391] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:37:13 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:37:13.324972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.335155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f043d-40fa-4012-8c41-2e2107280d52 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.394289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.394960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.395241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.395520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.396205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.398182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.485805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1199/2392] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:37:13 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:37:13.489893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.500961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f1b39-380f-40c9-b3bd-ca06bf0e30e6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.505907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.506576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.506859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.507135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.507868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.509492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.525420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1194/2393] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:37:13.527431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.538286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7728e-9a5f-4aaa-9c1c-72f0e88490f3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.544008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.544726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.545014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.545292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.546061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.548355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8979-7dbc-4d6c-a381-a381a8a280cd admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.558749 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1200/2394] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:37:13.562896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.573269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c464a-c7c7-464f-8e1a-ab72d17fec54 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.578026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.578933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.581788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.581788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] PATH_INFO: `/v3/projects/6b4b85f80e6c453e9fe6729b9a9a2570/users/d600deef3bbf4de1824f76f938891fc4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.594151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4b85f80e6c453e9fe6729b9a9a2570, user_id=d600deef3bbf4de1824f76f938891fc4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.597441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.725613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1195/2395] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:37:13 2022] PUT /identity/v3/projects/6b4b85f80e6c453e9fe6729b9a9a2570/users/d600deef3bbf4de1824f76f938891fc4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:37:13.789497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308ea9-c783-436f-b801-47d4470fbc36 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.789790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308ea9-c783-436f-b801-47d4470fbc36 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.790077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308ea9-c783-436f-b801-47d4470fbc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.821818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-21308ea9-c783-436f-b801-47d4470fbc36 None None] MFA Rules not processed for user `d600deef3bbf4de1824f76f938891fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:13.823040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-21308ea9-c783-436f-b801-47d4470fbc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:13.893774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1201/2396] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:37:13.904425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:13.906135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:13.928608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:13.965175 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d401-59ec-43cd-9097-f357d7e0ed27 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:13.969147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:13.970400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:13.970735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:13.971129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:13.984871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:13.984871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a83d0-7fcf-4651-877f-e6f2e3a5c2d4 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:13.990216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1196/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:37:23.890355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:24.106696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=17 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:24.230922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d6d8-b4c7-4096-9ab3-da2aaf80ec1a None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.231228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d6d8-b4c7-4096-9ab3-da2aaf80ec1a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.231487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d6d8-b4c7-4096-9ab3-da2aaf80ec1a None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.233532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1202/2398] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:37:24.239711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.240004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.240283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.260486 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] MFA Rules not processed for user `943602b745284e6b9a53ff217a9ee0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:24.262006 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:24.316930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d88fa-7273-42f3-b382-2a3c4169dda9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.317451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d88fa-7273-42f3-b382-2a3c4169dda9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.317793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d88fa-7273-42f3-b382-2a3c4169dda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.323233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1197/2399] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:24.335286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:24.335941 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-365d88fa-7273-42f3-b382-2a3c4169dda9 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:24.336823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:24.337147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-365d88fa-7273-42f3-b382-2a3c4169dda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:24.348606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28a47a-c91f-404c-a21f-d39252f11f10 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:24.352844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:24.353635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.355348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.355348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.362191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:24.363683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:24.372266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1198/2400] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:37:24.396933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1203/2401] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:37:24.410742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:24.412398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:24.462742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e881d-bf9a-4631-9e61-5629b9c9d655 None None] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.463050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e881d-bf9a-4631-9e61-5629b9c9d655 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.463309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e881d-bf9a-4631-9e61-5629b9c9d655 None None] PATH_INFO: `/` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.469994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1204/2402] 10.222.0.57 () {54 vars in 921 bytes} [Tue Jun 7 09:37:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:37:24.480240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.480635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.480998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.492448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5814dc9-b07e-4acc-977d-fdcd32669d98 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:24.498907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:24.499718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.500098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.500417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.510787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:24.512382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:24.514463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] MFA Rules not processed for user `dff6b630f6be464aa6cdcc73da587e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:24.515497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:24.521624 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1199/2403] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:37:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:37:24.588565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1205/2404] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Jun 7 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 07 09:37:24.617290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:24.617290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5af8-f421-44f5-99f9-a855e4856c9e service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:24.617290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:24.617290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:24.617788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:24.617788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:24.624035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:24.625506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f3b50-6f26-411b-8ee1-56387350ecbf service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:24.636173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1200/2405] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:37:27.450994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:27.518008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb16784-4793-4b45-9560-4cb58e67a12c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:27.522696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:27.523574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:27.523860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:27.524137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] PATH_INFO: `/v3/users/d600deef3bbf4de1824f76f938891fc4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:27.525425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d600deef3bbf4de1824f76f938891fc4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:27.527365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:27.712178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:27.725971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd600deef3bbf4de1824f76f938891fc4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:27.737447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd600deef3bbf4de1824f76f938891fc4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:27.740458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd600deef3bbf4de1824f76f938891fc4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:27.815735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd600deef3bbf4de1824f76f938891fc4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:27.816961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd600deef3bbf4de1824f76f938891fc4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:27.841278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1206/2406] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:37:27 2022] DELETE /identity/v3/users/d600deef3bbf4de1824f76f938891fc4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:37:27.849284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:27.849835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:27.849835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:27.879182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:27.880410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:27.971572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1201/2407] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:27.983880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:27.986011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:28.069385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f1be-997a-45a5-8320-7177fad2afdf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:28.076281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:28.082494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:28.082494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:28.082494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:28.088094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:28.089702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb959e4-d197-42d8-95d0-86f31e1feb71 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:28.104758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1207/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:37:29.195804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:29.197156 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:29.213471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d239d-4b23-4ab5-906d-d7810fa5f8d5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:29.220662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:29.221542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:29.221824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:29.222089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] PATH_INFO: `/v3/projects/6b4b85f80e6c453e9fe6729b9a9a2570` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:29.225422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4b85f80e6c453e9fe6729b9a9a2570)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:29.228005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:29.238022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:29.371426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4b85f80e6c453e9fe6729b9a9a2570', 'request_id': 'req-edaa877d-7310-4578-8dfd-3b46b60661dc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:29.379182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4b85f80e6c453e9fe6729b9a9a2570', 'request_id': 'req-edaa877d-7310-4578-8dfd-3b46b60661dc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:29.385040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4b85f80e6c453e9fe6729b9a9a2570', 'request_id': 'req-edaa877d-7310-4578-8dfd-3b46b60661dc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:29.462269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] The token cache is being invalidate because project 6b4b85f80e6c453e9fe6729b9a9a2570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:37:29.462571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:29.463238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:29.464901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1202/2409] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:37:29 2022] DELETE /identity/v3/projects/6b4b85f80e6c453e9fe6729b9a9a2570 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:37:38.096490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:39.458364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:46.412358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:46.412887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:46.412981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:46.430443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:37:46.431642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:46.489836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1208/2410] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:37:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:37:46.502290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:46.503760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:46.581740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa877d-7310-4578-8dfd-3b46b60661dc admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:46.586800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:46.587454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:46.587740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:46.588018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:46.597126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:46.598769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:46.607619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1203/2411] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:37:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:37:55.664925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:55.666331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:55.722148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a505155-c78e-4fbb-a435-b5a36cab5564 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:55.725741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:55.726645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:55.726938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:55.727219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:55.729158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:55.791126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:55.792558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:55.802196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1209/2412] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:37:56.604170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=32 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:59.250018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:59.251505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:59.325191 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb6f70-4309-4749-922f-520e7304d03e service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:59.333059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:59.339371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:59.339371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:59.339371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:59.339371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:59.353898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:59.356368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:59.371373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1204/2413] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:37:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:37:59.535761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:37:59.537262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:37:59.553201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9d75-eea9-4068-b188-6617b1311978 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:37:59.557938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:37:59.558840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:37:59.559126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:37:59.559408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] PATH_INFO: `/v3/users/0346f4e3855142059983083e71c2c0c3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:37:59.560621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0346f4e3855142059983083e71c2c0c3)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:37:59.562604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:37:59.794219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0346f4e3855142059983083e71c2c0c3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:59.796209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:37:59.802117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0346f4e3855142059983083e71c2c0c3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:59.807523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0346f4e3855142059983083e71c2c0c3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:59.888491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0346f4e3855142059983083e71c2c0c3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:59.889128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5da026f3-3baa-4ed3-a007-2df9196c3c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0346f4e3855142059983083e71c2c0c3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:37:59.906987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1210/2414] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:37:59 2022] DELETE /identity/v3/users/0346f4e3855142059983083e71c2c0c3 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:00.177128 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.189971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf9a25-a778-442e-9881-0dbe18513bb9 service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.249891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.257103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.257103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.257103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] PATH_INFO: `/v3/projects/1c21b1cbbd604d6d9c97ab7c3c788f3d` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.257103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c21b1cbbd604d6d9c97ab7c3c788f3d)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.257103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.363847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c21b1cbbd604d6d9c97ab7c3c788f3d', 'request_id': 'req-ad1d791a-b32d-40b5-a2ae-17d75928bc47'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:00.369902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c21b1cbbd604d6d9c97ab7c3c788f3d', 'request_id': 'req-ad1d791a-b32d-40b5-a2ae-17d75928bc47'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:00.375408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c21b1cbbd604d6d9c97ab7c3c788f3d', 'request_id': 'req-ad1d791a-b32d-40b5-a2ae-17d75928bc47'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:00.471099 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] The token cache is being invalidate because project 1c21b1cbbd604d6d9c97ab7c3c788f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:00.471313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:00.471929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ad1d791a-b32d-40b5-a2ae-17d75928bc47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:00.473510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1205/2415] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:00 2022] DELETE /identity/v3/projects/1c21b1cbbd604d6d9c97ab7c3c788f3d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:00.478681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.481844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.481844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.496855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:00.498038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:00.558182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1211/2416] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:00.566267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.566584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.566864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.585854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:00.587353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:00.595790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1206/2417] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:00.600005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.609966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f1db-ab41-41bd-9b22-ca3d3202fcca None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.614542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.615193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.615478 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.615752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.616467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:38:00.616803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.618455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.629852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1212/2418] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:38:00.633909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.643598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6eb6b-c6e3-49e1-870e-f47ae860919b None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.650457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.650457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.650457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.650457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.650896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.652172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.662732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:00.741868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1207/2419] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:00 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:00.745869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.755892 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0be1c-6b79-43d9-b4ee-8c095fffdc21 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.811140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.811799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.812086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.812365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.813100 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.815046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.917138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1213/2420] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:00 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:00.921174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.931786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906933c-3048-44b0-b35e-ecded9e8ace4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.936529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.937211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.937499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.937779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.938510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.940705 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.951178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1208/2421] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:00.955695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.965623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cf2fa-6ee9-45d9-9503-49c5d813926d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:00.970236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:00.970878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:00.971165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:00.971446 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:00.972165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:00.973805 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d15f19-9701-4e27-879c-196925f2a55d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:00.983854 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1214/2422] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:00.987698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:00.997710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99645a-db02-46b6-814e-45db5ee0ed25 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.002796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.003665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.003950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.004224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] PATH_INFO: `/v3/projects/877d8fea286d467db63e89488bb2a4cf/users/f66dec42ef964106b650044ffa105382/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.019519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877d8fea286d467db63e89488bb2a4cf, user_id=f66dec42ef964106b650044ffa105382, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.022231 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.106923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1209/2423] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:00 2022] PUT /identity/v3/projects/877d8fea286d467db63e89488bb2a4cf/users/f66dec42ef964106b650044ffa105382/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:01.116986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.117280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.117558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.145547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] MFA Rules not processed for user `f66dec42ef964106b650044ffa105382`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:01.146760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:01.166085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:01.209998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1215/2424] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:01.214495 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.276791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b1ff-7e66-489d-9573-dd40dff99070 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.281333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.281986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.282271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.282545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.283237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.285177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.364754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1210/2425] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:01 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:01.368950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.380368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab514c-8c4d-43a8-bb50-2101db3a87b4 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.434623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.435277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.435558 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.435830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.436497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.438423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.528960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1216/2426] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:01 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:01.532904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.543256 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c695fe7-85d2-4fd1-ba9e-c6a395038a55 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.548457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.549126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.549407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.549682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.550402 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.552033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.563504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1211/2427] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:01.568018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.578161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644665a-554a-4869-b59d-6de2d3a67245 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.582878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.583743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.584027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.584304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] PATH_INFO: `/v3/projects/1ae3cf4d163740888241188b3eaecf91/users/5b790b30efdf41bfab16d9abfad8a58c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.598786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae3cf4d163740888241188b3eaecf91, user_id=5b790b30efdf41bfab16d9abfad8a58c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.601730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.683461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1217/2428] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:01 2022] PUT /identity/v3/projects/1ae3cf4d163740888241188b3eaecf91/users/5b790b30efdf41bfab16d9abfad8a58c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:01.687505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.697481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a68-3acf-4674-bd82-08c18d32c669 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.753350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.754040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.754325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.754600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.755325 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.756965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f09df4-d8ca-404f-9ac8-8c6494d1eb74 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.766812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1212/2429] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:01.770937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.781115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa829b38-414d-4251-876a-6372ab1684e1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.786073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.786930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.787205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.787473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] PATH_INFO: `/v3/projects/1ae3cf4d163740888241188b3eaecf91/users/5b790b30efdf41bfab16d9abfad8a58c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.789448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae3cf4d163740888241188b3eaecf91, user_id=5b790b30efdf41bfab16d9abfad8a58c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.792045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:01.802610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1218/2430] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:01 2022] PUT /identity/v3/projects/1ae3cf4d163740888241188b3eaecf91/users/5b790b30efdf41bfab16d9abfad8a58c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:01.813445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.813796 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.814080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.839738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] MFA Rules not processed for user `5b790b30efdf41bfab16d9abfad8a58c`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:01.840983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:01.854024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:01.908357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1213/2431] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:01.919577 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:01.921360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:01.976913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee23e-e50d-4fb5-b373-c16cb246a2ff admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:01.980523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:01.981184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:01.981463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:01.981735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:01.991239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:01.992684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:02.001908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1219/2432] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:02.192570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:02.202332 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3278d5b-1fd4-470b-88f5-ed091f7561d5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:02.206418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:02.207050 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:02.207379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:02.207658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:02.265318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:02.267248 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:02.276254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1214/2433] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:11.997717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:12.272136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=50 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:24.561591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:24.563081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:24.620867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517a03-a64d-4bc9-8005-e9e2c09bdc49 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:24.624488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:24.625154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:24.625433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:24.625707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:24.627475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:24.697226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:24.698731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:24.709952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1220/2434] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:25.906670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:25.908224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:25.921105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b00f0-e327-412f-9833-61d10a61aeaf service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:25.925443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:25.926116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:25.926399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:25.926672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:25.928447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:25.942340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:25.943756 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:25.954714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1215/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:34.704671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=54 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:35.949884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=6 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:41.530523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:41.539536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4f26-5583-4a5a-b509-064e7180d6ec service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:41.543155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:41.543789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:41.544067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:41.544341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:41.546158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:41.608133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:41.609594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:41.619965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1221/2436] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:45.250116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:45.252522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:45.274436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf313d8-1e00-417e-9d5e-db4b40d17fc3 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:45.281621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:45.282363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:45.282669 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:45.282925 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] PATH_INFO: `/v3/users/2f32356bb0bd4234ba817db581b62aad` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:45.298522 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f32356bb0bd4234ba817db581b62aad)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:45.301162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:45.496997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f32356bb0bd4234ba817db581b62aad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:45.506425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f32356bb0bd4234ba817db581b62aad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:45.602411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f32356bb0bd4234ba817db581b62aad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:45.603031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f32356bb0bd4234ba817db581b62aad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:45.603649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-4ebb7a2b-13c5-42a9-8103-0b2a5bafcc84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f32356bb0bd4234ba817db581b62aad'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:45.622118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1216/2437] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:45 2022] DELETE /identity/v3/users/2f32356bb0bd4234ba817db581b62aad => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:45.862068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:45.872576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2a27f-46e9-4943-ba80-7c640134595e service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:45.939265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:45.940660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:45.940940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:45.941207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] PATH_INFO: `/v3/projects/50189b9cf3b145ba84495af8db63c41b` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:45.951469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50189b9cf3b145ba84495af8db63c41b)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:45.953409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.094924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50189b9cf3b145ba84495af8db63c41b', 'request_id': 'req-c48df04e-1553-4903-8300-cff3ad788685'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:46.100695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50189b9cf3b145ba84495af8db63c41b', 'request_id': 'req-c48df04e-1553-4903-8300-cff3ad788685'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:46.108856 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50189b9cf3b145ba84495af8db63c41b', 'request_id': 'req-c48df04e-1553-4903-8300-cff3ad788685'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:46.184265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] The token cache is being invalidate because project 50189b9cf3b145ba84495af8db63c41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:46.184549 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:46.185195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c48df04e-1553-4903-8300-cff3ad788685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:46.186716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1222/2438] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:45 2022] DELETE /identity/v3/projects/50189b9cf3b145ba84495af8db63c41b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:46.192252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.192552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.192865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.211801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:46.213053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:46.271889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1217/2439] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:46.279684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.279966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.280238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.297888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:46.299622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:46.307533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1223/2440] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:46.311661 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.321996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102f752-48b2-49aa-8527-2f361f618d10 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.326747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.327395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.327683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.327962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.328759 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:38:46.329094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.330736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.342349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1218/2441] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:38:46.346330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.355737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2544ca2-06f6-40a0-9357-98e7ab9ea5b7 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.360573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.361334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.362154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.362427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.363115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.365081 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.369115 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:46.495970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1224/2442] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/projects => generated 403 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:38:46.500232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.511023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f63b4-6ccc-49d2-bca1-81d348a50a24 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.555788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:46.567755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.568415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.568732 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.569005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.569681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.571581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.659631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1219/2443] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/users => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:46.663798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.674833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8309c-abfd-4880-b229-89746b98be20 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.679416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.680063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.680346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.680654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.681371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.682959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.692799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1225/2444] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:46.696753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.706690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1ca5-82ba-4cea-94bf-5ce50ebb132a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.711284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.711931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.712215 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.712492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.713246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.714823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91128323-de35-4299-a4e3-d33a53f0a416 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.726471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1220/2445] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:46.730147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:46.740531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ebea3-7a50-4c0f-a26e-5dfcf487b34e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:46.745176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:46.746092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.746324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.746625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] PATH_INFO: `/v3/projects/3acbede26f554f5db812805b6de0e7be/users/bb74784f42ff4dd48a222180c6cbf483/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.761031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acbede26f554f5db812805b6de0e7be, user_id=bb74784f42ff4dd48a222180c6cbf483, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:46.766383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:46.877345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1226/2446] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:46 2022] PUT /identity/v3/projects/3acbede26f554f5db812805b6de0e7be/users/bb74784f42ff4dd48a222180c6cbf483/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:46.887668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:46.892551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:46.892551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:46.917082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] MFA Rules not processed for user `bb74784f42ff4dd48a222180c6cbf483`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:46.918238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:46.987270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1221/2447] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:46.991744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.056711 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaba4c-e5bf-4440-a003-9a50a62baf8d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.061479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.062164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.062450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.062729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.063430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.065455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.227387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1227/2448] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:46 2022] POST /identity/v3/projects => generated 403 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:38:47.231463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.242470 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88acdd8-aa0d-45f2-b833-d18ec5b6d3a8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.300223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.300910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.301112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.301386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.302070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.303999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.397648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1222/2449] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:47 2022] POST /identity/v3/users => generated 308 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:47.401560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.411763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0dd8f-87ea-4702-8f85-9a722752b125 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.416403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.417113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.417396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.417669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.418388 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.419985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.430730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1228/2450] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:47.434921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.445986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190182fc-523c-44fe-8ec2-5d3d78048830 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.450605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.451491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.451752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.452121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] PATH_INFO: `/v3/projects/5a75f332a2334d44ae9c2c315c3846d5/users/803409dd36b540ceaac0fd37b89bb715/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.467525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a75f332a2334d44ae9c2c315c3846d5, user_id=803409dd36b540ceaac0fd37b89bb715, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.470304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.539408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1223/2451] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:47 2022] PUT /identity/v3/projects/5a75f332a2334d44ae9c2c315c3846d5/users/803409dd36b540ceaac0fd37b89bb715/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:47.543438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.553755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61557b-b99d-4829-aa0a-0b31c18a9e0b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.617176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.617777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.618064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.618511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.619123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.620756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffefc3c-9453-4db7-a680-c7e64f9f0262 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.631975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1229/2452] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:38:47.635939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.647051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63aeac-eb09-47a4-bed1-2ad6b5f1829f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.659247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.660167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.660451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.660776 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] PATH_INFO: `/v3/projects/5a75f332a2334d44ae9c2c315c3846d5/users/803409dd36b540ceaac0fd37b89bb715/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.662812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a75f332a2334d44ae9c2c315c3846d5, user_id=803409dd36b540ceaac0fd37b89bb715, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.665568 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.676619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1224/2453] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:47 2022] PUT /identity/v3/projects/5a75f332a2334d44ae9c2c315c3846d5/users/803409dd36b540ceaac0fd37b89bb715/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:47.686361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.686647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.686922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.715576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] MFA Rules not processed for user `803409dd36b540ceaac0fd37b89bb715`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:47.716847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:47.726467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:47.783205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1230/2454] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:47.794454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:47.796445 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:47.812048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:47.854482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d996120-48b6-4ea1-8704-83f268215af0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:47.858234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:47.858903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:47.859183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:47.859454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:47.869346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:47.870777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddf49e-5b57-4113-96ea-c9101d7943b0 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:47.880898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1225/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:47 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:48.105767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:48.167195 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24fea4-2413-41d9-b82c-7adef0d75573 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:48.171710 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:48.172643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:48.172933 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:48.173214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] PATH_INFO: `/v3/users/bb74784f42ff4dd48a222180c6cbf483` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:48.175017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb74784f42ff4dd48a222180c6cbf483)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:48.176984 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:48.375519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb74784f42ff4dd48a222180c6cbf483'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:48.382798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb74784f42ff4dd48a222180c6cbf483'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:48.383451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb74784f42ff4dd48a222180c6cbf483'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:48.459466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb74784f42ff4dd48a222180c6cbf483'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:48.460076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb74784f42ff4dd48a222180c6cbf483'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:48.478643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1231/2456] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:48 2022] DELETE /identity/v3/users/bb74784f42ff4dd48a222180c6cbf483 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:48.483097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:48.483586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:48.483701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:48.500375 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:48.501584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:48.559714 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1226/2457] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:48.569360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:48.570783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:48.602894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:48.629218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e708fe-5baa-4bf8-b4f1-8ab265b4d673 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:48.632907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:48.633557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:48.633878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:48.634155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:48.643840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:48.645360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:48.655127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1232/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:49.096372 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:49.097971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:49.110887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8013d6-217d-44ce-936f-0cbb753c6d9a None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:49.116355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:49.117280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:49.117564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:49.117840 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] PATH_INFO: `/v3/projects/3acbede26f554f5db812805b6de0e7be` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:49.119039 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acbede26f554f5db812805b6de0e7be)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:49.121010 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:49.306068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acbede26f554f5db812805b6de0e7be', 'request_id': 'req-5d0bd79e-d812-4d15-a852-6a5b4548c30e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.312399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acbede26f554f5db812805b6de0e7be', 'request_id': 'req-5d0bd79e-d812-4d15-a852-6a5b4548c30e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.318383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acbede26f554f5db812805b6de0e7be', 'request_id': 'req-5d0bd79e-d812-4d15-a852-6a5b4548c30e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.430516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] The token cache is being invalidate because project 3acbede26f554f5db812805b6de0e7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:49.430804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.431465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.433139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1227/2459] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:49 2022] DELETE /identity/v3/projects/3acbede26f554f5db812805b6de0e7be => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:49.437095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:49.446889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc58ae-63fa-43a0-83d7-5130a7d504be service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:49.503336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:49.504305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:49.504617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:49.504898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] PATH_INFO: `/v3/users/803409dd36b540ceaac0fd37b89bb715` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:49.506085 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803409dd36b540ceaac0fd37b89bb715)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:49.508155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:49.761076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803409dd36b540ceaac0fd37b89bb715'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.767975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803409dd36b540ceaac0fd37b89bb715'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.768670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803409dd36b540ceaac0fd37b89bb715'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.824995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803409dd36b540ceaac0fd37b89bb715'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.825623 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-3e4f0da2-dae7-4d82-9b19-df7d15194784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803409dd36b540ceaac0fd37b89bb715'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:49.843246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1233/2460] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:49 2022] DELETE /identity/v3/users/803409dd36b540ceaac0fd37b89bb715 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:50.249560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.251145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:50.264200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd79e-d812-4d15-a852-6a5b4548c30e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.297737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:50.321563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.322417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.322698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.322972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] PATH_INFO: `/v3/projects/5a75f332a2334d44ae9c2c315c3846d5` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.324153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a75f332a2334d44ae9c2c315c3846d5)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.326103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.426786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a75f332a2334d44ae9c2c315c3846d5', 'request_id': 'req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:50.432916 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a75f332a2334d44ae9c2c315c3846d5', 'request_id': 'req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:50.438359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a75f332a2334d44ae9c2c315c3846d5', 'request_id': 'req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:50.516085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] The token cache is being invalidate because project 5a75f332a2334d44ae9c2c315c3846d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:50.516085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:50.516627 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-942a71d7-2bc6-47a4-87f6-5b7ced0119c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:50.518261 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1228/2461] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:50 2022] DELETE /identity/v3/projects/5a75f332a2334d44ae9c2c315c3846d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:50.523235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.523672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.523768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.541355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:50.542550 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:50.587434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:50.603248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1234/2462] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:50.612805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.613098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.613378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.631394 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:50.632600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:50.641696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1229/2463] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:50.646114 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.656046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46744592-3497-47ad-b0af-a304a0638e36 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.661364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.662039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.662324 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.662600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.663350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:38:50.663664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.665674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.678393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1235/2464] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:38:50.682748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.691919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798ba0-ddc7-4e02-b3f1-4a37d648832d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.696612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.700924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.700924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.700924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.700924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.700924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.780009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1230/2465] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:50 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:50.784626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.794878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc29dc-375f-428a-8f9b-0143859e3efe admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.852107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.852797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.853090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.853368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.854048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.855968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.911472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1236/2466] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:50 2022] POST /identity/v3/users => generated 282 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:50.915996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.929322 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28971ed5-2845-4108-a620-f950bf1b20e4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.934772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.935424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.935709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.935985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.936748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.938338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.948657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1231/2467] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:50.952603 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.963087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8eb0d-ba21-43ef-8449-00ec0f8b28d9 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:50.967725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:50.968372 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:50.968685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:50.968965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:50.969685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:50.972004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58700f8d-cc48-49ef-ba4d-526c1176d329 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:50.982048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1237/2468] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:50.985963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:50.995907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525946-94cf-4113-b2b1-60736f09004f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.000537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.001442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.001727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.002005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] PATH_INFO: `/v3/projects/2a7f6539589f4811b8457b4150d0036b/users/571ddc762b844220abae94462e211356/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.016730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7f6539589f4811b8457b4150d0036b, user_id=571ddc762b844220abae94462e211356, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.019909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.071194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1232/2469] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:50 2022] PUT /identity/v3/projects/2a7f6539589f4811b8457b4150d0036b/users/571ddc762b844220abae94462e211356/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:38:51.081760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.082049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.082331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.108657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] MFA Rules not processed for user `571ddc762b844220abae94462e211356`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:51.109860 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:51.171968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1238/2470] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:51.176728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.240345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bd323-2f17-4ec7-b204-6beb958f38fb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.245108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.245764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.246047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.246321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.247032 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.249012 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.253544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:51.335311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1233/2471] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:51 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:51.339506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.350489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ca7f7-a7b4-4f75-bb75-d92b50d64496 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.393409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:51.408070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.408774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.409060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.409335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.410054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.412030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.505963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1239/2472] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:51 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:51.509774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.520330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d047e9-361e-46ce-b0ad-575fa458f926 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.525135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.525785 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.526071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.526369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.527117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.528767 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.541126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1234/2473] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:51.546448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.557184 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aead34d-ea34-41e1-bd4d-2fb980224e39 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.561897 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.562765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.563053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.563332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] PATH_INFO: `/v3/projects/d458d40c1ce645358aa056f2592852b9/users/22b1ed8875c74c60a0c0f255f911fc84/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.578504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d458d40c1ce645358aa056f2592852b9, user_id=22b1ed8875c74c60a0c0f255f911fc84, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.581203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.677735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1240/2474] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:51 2022] PUT /identity/v3/projects/d458d40c1ce645358aa056f2592852b9/users/22b1ed8875c74c60a0c0f255f911fc84/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:51.681644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.692497 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273ed8e-2e73-4fca-8ded-36cbe8e3fb0b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.747898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.748550 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.748866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.749139 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.749858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.751471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3b008-2f84-43e3-81e5-ce0ef8723c98 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.761583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1235/2475] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:38:51.765602 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.776003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e2b8-2f50-4f17-b52c-8a1f351f3d3e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.780633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.781504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.781788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.782064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] PATH_INFO: `/v3/projects/d458d40c1ce645358aa056f2592852b9/users/22b1ed8875c74c60a0c0f255f911fc84/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.784023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d458d40c1ce645358aa056f2592852b9, user_id=22b1ed8875c74c60a0c0f255f911fc84, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.786671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.796935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1241/2476] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:51 2022] PUT /identity/v3/projects/d458d40c1ce645358aa056f2592852b9/users/22b1ed8875c74c60a0c0f255f911fc84/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:38:51.807501 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.807793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.808071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.834357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] MFA Rules not processed for user `22b1ed8875c74c60a0c0f255f911fc84`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:51.835553 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:51.901876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1236/2477] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:51.914005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:51.915790 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:51.972105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57c2b7-7635-45cc-b0e8-e8bc49aa58e4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:51.975988 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:51.976642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:51.976928 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:51.977206 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:51.986203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:51.988179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3ee1-122d-40b8-a861-e1bdf8bb8494 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:51.997532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1242/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:52.853373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:52.911427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:52.915466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e226-c42b-48be-adeb-320b54744e7f None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:52.919969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:52.920859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:52.921149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:52.921420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] PATH_INFO: `/v3/users/571ddc762b844220abae94462e211356` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:52.922635 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571ddc762b844220abae94462e211356)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:52.924549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:53.101729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571ddc762b844220abae94462e211356'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.108793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571ddc762b844220abae94462e211356'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.201839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571ddc762b844220abae94462e211356'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.202452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571ddc762b844220abae94462e211356'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.203072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571ddc762b844220abae94462e211356'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.220619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1237/2479] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:52 2022] DELETE /identity/v3/users/571ddc762b844220abae94462e211356 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:53.225484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:53.225963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:53.226077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:53.244486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:53.245714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:53.303831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1243/2480] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:53.313423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:53.315441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:53.379531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202103-b7eb-4d47-a3f5-7a808208b645 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:53.383089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:53.383723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:53.384021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:53.384293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:53.393369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:53.394791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:53.404200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1238/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:53.809754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:53.811212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:53.834866 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295723b0-a631-47eb-ae5c-8efcefcad259 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:53.846464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:53.847404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:53.847761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:53.848108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] PATH_INFO: `/v3/projects/2a7f6539589f4811b8457b4150d0036b` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:53.849340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7f6539589f4811b8457b4150d0036b)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:53.851567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:53.941874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:53.951093 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7f6539589f4811b8457b4150d0036b', 'request_id': 'req-016a8bd5-17bc-4104-a457-3d817c5be5c8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.957038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7f6539589f4811b8457b4150d0036b', 'request_id': 'req-016a8bd5-17bc-4104-a457-3d817c5be5c8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:53.963166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7f6539589f4811b8457b4150d0036b', 'request_id': 'req-016a8bd5-17bc-4104-a457-3d817c5be5c8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.041430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] The token cache is being invalidate because project 2a7f6539589f4811b8457b4150d0036b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:54.041721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.042350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.043910 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1244/2482] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:53 2022] DELETE /identity/v3/projects/2a7f6539589f4811b8457b4150d0036b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:54.048001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:54.058045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5799e-65c8-47f0-a86a-e521dab30ece service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:54.115947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:54.116835 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:54.117118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:54.117390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] PATH_INFO: `/v3/users/22b1ed8875c74c60a0c0f255f911fc84` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:54.118587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b1ed8875c74c60a0c0f255f911fc84)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:54.120768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:54.149935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:54.461577 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b1ed8875c74c60a0c0f255f911fc84'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.468348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b1ed8875c74c60a0c0f255f911fc84'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.573965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b1ed8875c74c60a0c0f255f911fc84'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.574593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b1ed8875c74c60a0c0f255f911fc84'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.575275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-8336adcc-16a6-4aa3-967d-2c70cb63b07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b1ed8875c74c60a0c0f255f911fc84'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:54.592700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1239/2483] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:54 2022] DELETE /identity/v3/users/22b1ed8875c74c60a0c0f255f911fc84 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:55.038900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.040349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:55.054502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a8bd5-17bc-4104-a457-3d817c5be5c8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.115258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.116123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.116408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.116716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] PATH_INFO: `/v3/projects/d458d40c1ce645358aa056f2592852b9` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.117912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d458d40c1ce645358aa056f2592852b9)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.119873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.225970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd458d40c1ce645358aa056f2592852b9', 'request_id': 'req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:55.231587 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd458d40c1ce645358aa056f2592852b9', 'request_id': 'req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:55.237596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd458d40c1ce645358aa056f2592852b9', 'request_id': 'req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:55.322323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] The token cache is being invalidate because project d458d40c1ce645358aa056f2592852b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:55.322460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:55.323137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-07ce94b6-49a6-48ed-9bdf-a8e8fd5cd90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:55.325011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1245/2484] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:55 2022] DELETE /identity/v3/projects/d458d40c1ce645358aa056f2592852b9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:55.329981 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.330509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.330632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.350496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:55.351699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:55.412007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1240/2485] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:38:55.420843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.421132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.421410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.439275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:55.440483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:55.448399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1246/2486] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:55.452822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.462928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e414ff0-e188-4c2b-a025-0424b5dad608 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.467513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.468167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.468451 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.468762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.469660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:38:55.469861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.471415 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.483815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1241/2487] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:38:55.488117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.497462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b8d94-9f4c-4419-b915-5d8efa4d7bd8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.502256 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.502898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.503319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.503436 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.504126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.506072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.590301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1247/2488] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:55 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:55.591481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.602438 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d367e70-db7f-467a-b02b-6216fa8e94f6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.662901 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.663611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.663895 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.664168 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.664928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.666883 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.759185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1242/2489] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:55 2022] POST /identity/v3/users => generated 301 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:38:55.762335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.773743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26bb23-9b1a-4612-a932-19c184e16c86 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.778520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.779247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.779534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.779883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.780531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.782191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.792748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1248/2490] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:55.796812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.807178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87f46a-0a6b-43f2-ae5f-bc31f48a854f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.811898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.813683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.813683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.813683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.813965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.815518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c233e74-190d-4008-82f1-b8bc0a6edc2c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.827821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1243/2491] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:55.832670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:55.842745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d335ad-09cb-4b25-aea8-00d42bb115ef admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:55.847476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:55.848345 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.848657 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.848936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] PATH_INFO: `/v3/projects/8495d87af9db4fbf86963d7f76146557/users/5df6f59e91ed4e24ae111fea918f9aab/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:55.860442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:55.864418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8495d87af9db4fbf86963d7f76146557, user_id=5df6f59e91ed4e24ae111fea918f9aab, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:55.867165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:55.985363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1249/2492] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:55 2022] PUT /identity/v3/projects/8495d87af9db4fbf86963d7f76146557/users/5df6f59e91ed4e24ae111fea918f9aab/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:55.995993 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:55.996284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:55.996561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.003355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:56.031974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] MFA Rules not processed for user `5df6f59e91ed4e24ae111fea918f9aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:56.033237 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:56.097986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1244/2493] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:56.102635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.165997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c745b35-7e55-4803-9227-ab8b1ca6a21c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.170978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.171648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.171930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.172208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.173033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.174950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.329666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1250/2494] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:38:56 2022] POST /identity/v3/projects => generated 399 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:56.333588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.343984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c157a5-8fd9-4e52-8715-0712e4c24f35 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.403494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.403494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.403494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.403494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.403862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.404704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.488134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1245/2495] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:38:56 2022] POST /identity/v3/users => generated 306 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:38:56.492533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.503021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb13f94-661d-45f7-a980-20732126d334 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.508743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.512407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.512407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.512407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.512407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.512407 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.526691 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1251/2496] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:38:56.528095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.543595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794c72b-29cb-482c-9412-af9d27bd4ec3 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.543595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.544183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.544465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.544798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] PATH_INFO: `/v3/projects/060ca482dc9e464a87a5e244ed9a7d73/users/b1be777319894e678108697626eeefd8/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.560329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] RBAC: Authorizing `identity:create_grant(project_id=060ca482dc9e464a87a5e244ed9a7d73, user_id=b1be777319894e678108697626eeefd8, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.563016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.666060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1246/2497] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:56 2022] PUT /identity/v3/projects/060ca482dc9e464a87a5e244ed9a7d73/users/b1be777319894e678108697626eeefd8/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:38:56.670270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.680565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5d03-b6c9-482b-8eb8-f9ceb80aec04 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.740379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.741105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.741460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.741643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.742364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.743978 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b5a1b-0cc3-4efd-8911-a74338f67691 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.756687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1252/2498] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:38:56.758620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.769185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1f08-e373-4fe4-924d-f02f3bd8937d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.774505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e359-1105-43e1-a321-864383668701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.775254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e359-1105-43e1-a321-864383668701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.775536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e359-1105-43e1-a321-864383668701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.775812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e359-1105-43e1-a321-864383668701 admin admin] PATH_INFO: `/v3/projects/060ca482dc9e464a87a5e244ed9a7d73/users/b1be777319894e678108697626eeefd8/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.777948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e359-1105-43e1-a321-864383668701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060ca482dc9e464a87a5e244ed9a7d73, user_id=b1be777319894e678108697626eeefd8, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.780706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e359-1105-43e1-a321-864383668701 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:56.792871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1247/2499] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:38:56 2022] PUT /identity/v3/projects/060ca482dc9e464a87a5e244ed9a7d73/users/b1be777319894e678108697626eeefd8/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:56.807699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.808075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.808448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.835836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] MFA Rules not processed for user `b1be777319894e678108697626eeefd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:56.837070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:56.882851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:56.904427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1253/2500] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:56.918329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e359-1105-43e1-a321-864383668701 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:56.918329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0032e359-1105-43e1-a321-864383668701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:56.971121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:56.977199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e359-1105-43e1-a321-864383668701 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:56.981162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:56.981902 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:56.982190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:56.982466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:56.992075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:56.993548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:57.003373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1248/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:38:57.099049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:57.100870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:57.165879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a35d-1b80-4c87-b722-e43c8976057b None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:57.170959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:57.171434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:57.171718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:57.171996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] PATH_INFO: `/v3/users/f66dec42ef964106b650044ffa105382` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:57.173281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66dec42ef964106b650044ffa105382)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:57.175202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:57.284790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:57.293991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c31c2-9e92-4511-8221-4a66c0fb93c4 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:57.297944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:57.298588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:57.298892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:57.299167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:57.357506 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:57.358954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7986f2c-5933-4bab-9c51-2ec84b9a67d2 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:57.370196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1249/2502] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:57.383452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66dec42ef964106b650044ffa105382'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:57.391547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66dec42ef964106b650044ffa105382'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:57.392251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66dec42ef964106b650044ffa105382'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:57.488048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66dec42ef964106b650044ffa105382'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:57.488880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66dec42ef964106b650044ffa105382'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:57.507722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1254/2503] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:57 2022] DELETE /identity/v3/users/f66dec42ef964106b650044ffa105382 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:57.512166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:57.512460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:57.512775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:57.530176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:57.531374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:57.590351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1250/2504] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:57.601016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:57.602420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:57.662724 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471bf8-411b-467b-9ebc-0bf14273e2b0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:57.666466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:57.667111 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:57.667482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:57.667762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:57.676900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:57.678327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:57.688049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1255/2505] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:38:58.301300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:58.304291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:58.318647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c6ed-7af2-4fca-a9cf-82e15e6a9161 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:58.346819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:58.348141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:58.348499 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:58.348848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] PATH_INFO: `/v3/projects/877d8fea286d467db63e89488bb2a4cf` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:58.350460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=877d8fea286d467db63e89488bb2a4cf)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:58.352473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:58.450086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877d8fea286d467db63e89488bb2a4cf', 'request_id': 'req-1717febd-6930-411b-99de-8904f2a66135'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.456556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877d8fea286d467db63e89488bb2a4cf', 'request_id': 'req-1717febd-6930-411b-99de-8904f2a66135'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.462354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877d8fea286d467db63e89488bb2a4cf', 'request_id': 'req-1717febd-6930-411b-99de-8904f2a66135'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.534605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] The token cache is being invalidate because project 877d8fea286d467db63e89488bb2a4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:58.535762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.535762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.538102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1251/2506] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:58 2022] DELETE /identity/v3/projects/877d8fea286d467db63e89488bb2a4cf => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:58.541812 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:58.551832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f787aea-184d-4d4a-a38f-b4f4e4da349f service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:58.566007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:58.638583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:58.639624 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:58.639913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:58.643849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] PATH_INFO: `/v3/users/5b790b30efdf41bfab16d9abfad8a58c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:58.643849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b790b30efdf41bfab16d9abfad8a58c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:58.643849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:58.897194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b790b30efdf41bfab16d9abfad8a58c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.909755 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b790b30efdf41bfab16d9abfad8a58c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:58.910512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b790b30efdf41bfab16d9abfad8a58c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.031681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b790b30efdf41bfab16d9abfad8a58c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.032315 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-81604ee1-813c-4277-9958-09fecb034dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b790b30efdf41bfab16d9abfad8a58c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.055297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1256/2507] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:38:58 2022] DELETE /identity/v3/users/5b790b30efdf41bfab16d9abfad8a58c => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:59.586843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:38:59.588863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:59.604686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717febd-6930-411b-99de-8904f2a66135 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:38:59.620462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:38:59.673708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:38:59.674723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:59.675018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:59.675294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] PATH_INFO: `/v3/projects/1ae3cf4d163740888241188b3eaecf91` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:59.676573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae3cf4d163740888241188b3eaecf91)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:38:59.678610 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:38:59.793384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae3cf4d163740888241188b3eaecf91', 'request_id': 'req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.800243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae3cf4d163740888241188b3eaecf91', 'request_id': 'req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.806007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae3cf4d163740888241188b3eaecf91', 'request_id': 'req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.888228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] The token cache is being invalidate because project 1ae3cf4d163740888241188b3eaecf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:38:59.891500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.891500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0d5a9aa7-d0ec-4d28-8d02-1a20310e3f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:38:59.891500 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1252/2508] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:38:59 2022] DELETE /identity/v3/projects/1ae3cf4d163740888241188b3eaecf91 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:38:59.896341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:59.896886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:59.896987 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:38:59.917088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:38:59.917088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:38:59.981160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1257/2509] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:38:59.989912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:38:59.990331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:38:59.990709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.015467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:00.016757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:00.025823 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1253/2510] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:00.030380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.044671 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7811bd5-080b-4b6d-b5f9-78a3419ded35 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.049802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.050631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.050966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.051299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.052955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:00.053367 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.055096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.074216 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1258/2511] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:00.076693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.086434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae2db6-71e9-4a55-88ff-545ecfd97bd5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.091192 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.091848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.092132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.092498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.093188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.098091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.233440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1254/2512] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:00 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:00.237521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.248166 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3e90-be92-4b3c-bca0-5e3766c91b12 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.257082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:00.311794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.312110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.312392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.312699 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.313400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.315516 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.388378 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1259/2513] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:00 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:00.389613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.400807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d9e3-fad0-4416-83fb-34ebd04abddd admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.405412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.406068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.406344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.406651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.407370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.408986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.421029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1255/2514] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:00.425176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.435920 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912d528-94c9-43ab-808a-f4ccd51bd601 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.440644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.441299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.441682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.441961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.442683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.444286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc36ef7-99b4-44d3-bea4-160e59b297dc admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.454035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1260/2515] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:00.458604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.468567 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53d0f-b7bf-47ea-b743-ec784d2333fb admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.473182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.476919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.476919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.476919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] PATH_INFO: `/v3/projects/8af4ee94483548de918820841c0b1546/users/8d1d999bc7454469ba7e6afe8d6092e9/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.490104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af4ee94483548de918820841c0b1546, user_id=8d1d999bc7454469ba7e6afe8d6092e9, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.492814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.551011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1256/2516] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:00 2022] PUT /identity/v3/projects/8af4ee94483548de918820841c0b1546/users/8d1d999bc7454469ba7e6afe8d6092e9/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:00.562056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.562352 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.562636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.589067 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] MFA Rules not processed for user `8d1d999bc7454469ba7e6afe8d6092e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:00.590273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:00.654771 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1261/2517] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:00.659541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.701414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:00.722423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808cae-20bf-4057-bd28-058f471cae06 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.728097 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.728790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.729078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.729374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.730059 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.731959 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:00.862482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1257/2518] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:00 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:00.869302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:00.880995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059fdd4-d565-4a5a-b10f-93fa7fed0bf9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:00.944335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:00.945186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:00.945534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:00.946024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:00.946796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:00.948825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.156737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1262/2519] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:00 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:01.160847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.171206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3799ec-68a5-4864-ab50-3fa00bb591bf admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.175838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.176489 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.177297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.177689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.178437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:01.180030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.190930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1258/2520] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:01.195609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.205564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720428f-a13b-4157-8422-4620369e367a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.210159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.211044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.211329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.211606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] PATH_INFO: `/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e/users/85de6c71742f4513ab48bd5e5f09c373/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.220188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:01.229311 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1a49af6d9247f59ee6e5d83c2bfe3e, user_id=85de6c71742f4513ab48bd5e5f09c373, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:01.232614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.328295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1263/2521] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:01 2022] PUT /identity/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e/users/85de6c71742f4513ab48bd5e5f09c373/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:01.336887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.347449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfd5d-e3be-4436-8f6d-250ff559353e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.408946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.408946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.408946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.408946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.409437 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:01.410379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc3176-2213-4243-939b-c470cfd35a30 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.420515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1259/2522] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:01.424676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.436275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02784d7-654a-4688-bcd3-ac5bd5357cf8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.441430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.442898 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.443196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.443485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] PATH_INFO: `/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e/users/85de6c71742f4513ab48bd5e5f09c373/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.445524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1a49af6d9247f59ee6e5d83c2bfe3e, user_id=85de6c71742f4513ab48bd5e5f09c373, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:01.448173 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.459409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1264/2523] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:01 2022] PUT /identity/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e/users/85de6c71742f4513ab48bd5e5f09c373/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:01.469648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.469937 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.470219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.497462 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] MFA Rules not processed for user `85de6c71742f4513ab48bd5e5f09c373`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:01.498700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:01.568955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1260/2524] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:01.581143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.582857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:01.641039 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dfa4-0841-4f1d-b327-9194448ad0a2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.644707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.645374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.645764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.646041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.656227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:01.659458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:01.669260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1265/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:01.924074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:01.934596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d0777-f8f3-4800-bb7c-2cda6c43ff66 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:01.938188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:01.938822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:01.939105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:01.939384 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:01.969537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:02.001259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:02.007203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368167bf-62ed-48ea-aacd-86fe503aa7da service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:02.017488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1261/2526] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:03.693399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:03.694857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:03.763198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d8f0-5ec4-432e-bab4-2a5a2130b9f6 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:03.767971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:03.769604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:03.769604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:03.769604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] PATH_INFO: `/v3/users/5df6f59e91ed4e24ae111fea918f9aab` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:03.770643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df6f59e91ed4e24ae111fea918f9aab)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:03.772663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:04.026831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df6f59e91ed4e24ae111fea918f9aab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.033627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df6f59e91ed4e24ae111fea918f9aab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.034282 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df6f59e91ed4e24ae111fea918f9aab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.116416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df6f59e91ed4e24ae111fea918f9aab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.117130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df6f59e91ed4e24ae111fea918f9aab'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.128750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:04.135363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1266/2527] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:03 2022] DELETE /identity/v3/users/5df6f59e91ed4e24ae111fea918f9aab => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:04.139885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:04.140363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:04.140481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:04.158122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:04.159582 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:04.219178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1262/2528] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:04.229965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:04.231421 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:04.291490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318538f-aa11-4f2b-acb4-d2a2215600c0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:04.295705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:04.296839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:04.296839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:04.297158 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:04.306735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:04.308210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:04.319258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1267/2529] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:04.523475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:04.536206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf24dc-0f6f-459a-8846-c53e269deeb7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:04.538473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:04.539631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:04.540083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:04.540427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:04.542752 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:04.556265 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:04.559149 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:04.568615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1263/2530] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:04.760048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:04.761809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:04.776608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f131c-e78d-40ff-9f33-5dda4bef4110 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:04.781573 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:04.782543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:04.782938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:04.783362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] PATH_INFO: `/v3/projects/8495d87af9db4fbf86963d7f76146557` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:04.784736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8495d87af9db4fbf86963d7f76146557)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:04.787119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:04.941390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8495d87af9db4fbf86963d7f76146557', 'request_id': 'req-109dd171-ac88-44b0-8940-52992423cde6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.948017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8495d87af9db4fbf86963d7f76146557', 'request_id': 'req-109dd171-ac88-44b0-8940-52992423cde6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:04.954618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8495d87af9db4fbf86963d7f76146557', 'request_id': 'req-109dd171-ac88-44b0-8940-52992423cde6'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.053479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] The token cache is being invalidate because project 8495d87af9db4fbf86963d7f76146557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:05.058318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.058318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.058318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1268/2531] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:04 2022] DELETE /identity/v3/projects/8495d87af9db4fbf86963d7f76146557 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:05.060337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:05.070923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a720f-a4ec-4595-a56a-e7f82396358d service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:05.127998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:05.128897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:05.129182 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:05.129460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] PATH_INFO: `/v3/users/b1be777319894e678108697626eeefd8` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:05.131413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1be777319894e678108697626eeefd8)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:05.133435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:05.331349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1be777319894e678108697626eeefd8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.340855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1be777319894e678108697626eeefd8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.345822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:05.402184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1be777319894e678108697626eeefd8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.402917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1be777319894e678108697626eeefd8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.404683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7ebe1cca-1a24-4560-9f2b-ee95dd9332b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1be777319894e678108697626eeefd8'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:05.428227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1264/2532] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:05 2022] DELETE /identity/v3/users/b1be777319894e678108697626eeefd8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:06.120088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:06.122270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:06.138689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd171-ac88-44b0-8940-52992423cde6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:06.201385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:06.202329 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.202617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.202894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] PATH_INFO: `/v3/projects/060ca482dc9e464a87a5e244ed9a7d73` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.204134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060ca482dc9e464a87a5e244ed9a7d73)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:06.206159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:06.233564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:06.361205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060ca482dc9e464a87a5e244ed9a7d73', 'request_id': 'req-7af70935-9d66-4b55-84f0-57422f746291'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:06.370221 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060ca482dc9e464a87a5e244ed9a7d73', 'request_id': 'req-7af70935-9d66-4b55-84f0-57422f746291'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:06.373589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060ca482dc9e464a87a5e244ed9a7d73', 'request_id': 'req-7af70935-9d66-4b55-84f0-57422f746291'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:06.498955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] The token cache is being invalidate because project 060ca482dc9e464a87a5e244ed9a7d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:06.499246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:06.499970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7af70935-9d66-4b55-84f0-57422f746291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:06.504849 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1269/2533] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:06 2022] DELETE /identity/v3/projects/060ca482dc9e464a87a5e244ed9a7d73 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:06.506784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.507235 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.507750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.528100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:06.529965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:06.593918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1265/2534] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:06.602684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.609031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.609031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.626025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:06.627295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:06.635616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1270/2535] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:06.640354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:06.650948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321449e-b360-4153-9472-b29ae9d2d388 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:06.656518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:06.657525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.657806 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.658081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.658808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:06.659117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:06.660908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:06.673822 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1266/2536] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:06.678172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:06.687489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b16f51-6c76-4572-aaff-9314c0557cef None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:06.692905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:06.693632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.693916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.695296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.695296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:06.696880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:06.815201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1271/2537] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:06 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:06.819769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:06.830900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57db84-fea3-41b8-ac1a-40504690831e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:06.891085 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:06.891860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:06.892146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:06.892416 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:06.893214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:06.895145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.034978 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1267/2538] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:06 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:07.038883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.049800 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dfaf1-44b8-42ce-980c-8491336fc7ea admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.054813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.055599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.055957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.056340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.057186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.058967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.071240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1272/2539] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:07.075464 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.086534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df940fb-b0b8-4768-b5d9-f1012dbc7878 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.091302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.091986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.092267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.092541 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.093307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.094909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2c192-05e4-4386-af45-490e7758e060 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.108245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1268/2540] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:07.111921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.126861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba7de-fbca-4eef-8b92-c8b5f792506d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.135043 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.135965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.136249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.136523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] PATH_INFO: `/v3/projects/cb846b3a2be74349bdfdb9d3fdd3ed61/users/e230316af1de45dd9711446916b179bc/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.152214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb846b3a2be74349bdfdb9d3fdd3ed61, user_id=e230316af1de45dd9711446916b179bc, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.154811 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.236513 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1273/2541] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:07 2022] PUT /identity/v3/projects/cb846b3a2be74349bdfdb9d3fdd3ed61/users/e230316af1de45dd9711446916b179bc/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:07.247446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.247742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.248027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.276825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] MFA Rules not processed for user `e230316af1de45dd9711446916b179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:07.278055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:07.315207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:07.350889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1269/2542] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:07.355401 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.426850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591866-dc48-4517-8f09-add24726542a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.433887 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.434625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.434943 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.435295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.436016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.438313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.557299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1274/2543] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:07 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:07.558466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.570367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdfcdb-b25e-494b-b30b-b155f2655873 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.635708 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.636391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.636725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.636991 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.637689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.639712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.803645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1270/2544] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:07 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:07.809462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.821167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c8fa5-6c95-4b6e-a938-167c4322f433 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.827400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.828201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.828485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.828806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.830097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.831732 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.847265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1275/2545] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:07.848604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:07.860391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f15ef-a9af-4266-a58f-67c59ef8df98 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:07.865647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:07.866531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:07.866813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:07.867087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] PATH_INFO: `/v3/projects/e0646706687a45fc95eb70c6deadd1a2/users/524d8335c8a34a99aa3108091ad79184/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:07.887345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0646706687a45fc95eb70c6deadd1a2, user_id=524d8335c8a34a99aa3108091ad79184, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:07.888225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:07.991352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1271/2546] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:07 2022] PUT /identity/v3/projects/e0646706687a45fc95eb70c6deadd1a2/users/524d8335c8a34a99aa3108091ad79184/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:07.995437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.011765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f242-7cb3-445d-b314-0436adea1b6f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:08.038982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:08.080110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:08.080867 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.081153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.081433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.082164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:08.084297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbfd89-3d10-4a5e-b57c-198d3bc9e28f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:08.094226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1276/2547] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:08.098389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.109225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b2cf7-c0a1-4c9a-ace6-fab438afdde7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:08.114371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:08.116653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.116653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.116653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] PATH_INFO: `/v3/projects/e0646706687a45fc95eb70c6deadd1a2/users/524d8335c8a34a99aa3108091ad79184/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.118594 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0646706687a45fc95eb70c6deadd1a2, user_id=524d8335c8a34a99aa3108091ad79184, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:08.121400 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:08.132666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1272/2548] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:08 2022] PUT /identity/v3/projects/e0646706687a45fc95eb70c6deadd1a2/users/524d8335c8a34a99aa3108091ad79184/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:08.143646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.144112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.144310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.171083 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] MFA Rules not processed for user `524d8335c8a34a99aa3108091ad79184`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:08.172191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:08.258703 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1277/2549] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:08.266276 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.268045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:08.328355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef45eb-2e54-4286-aa1d-455381b3897c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:08.338286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:08.338948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.339233 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.339510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.350688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:08.352129 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:08.362348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1273/2550] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:08.493815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.535934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.537490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:08.562904 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cd94c-343f-4d26-9033-75d72e528978 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:08.567475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:08.568965 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.569254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.569532 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] PATH_INFO: `/v3/users/e230316af1de45dd9711446916b179bc` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.570733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e230316af1de45dd9711446916b179bc)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:08.571559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:08.572757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:08.605355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e30c-0853-49b7-8e42-9c7c9cf690a5 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:08.610826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:08.611692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.612082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.612362 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] PATH_INFO: `/v3/users/06e30bbe21af428faa5ec0eab414abf5` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.626066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e30bbe21af428faa5ec0eab414abf5)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:08.628003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:08.768872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:08.770391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e230316af1de45dd9711446916b179bc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.777029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e230316af1de45dd9711446916b179bc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.777672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e230316af1de45dd9711446916b179bc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.794315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e30bbe21af428faa5ec0eab414abf5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.801199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e30bbe21af428faa5ec0eab414abf5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.859252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e230316af1de45dd9711446916b179bc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.859882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-739edfae-9c9b-4ead-b469-0a58a90c960d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e230316af1de45dd9711446916b179bc'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.878601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1278/2551] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:08 2022] DELETE /identity/v3/users/e230316af1de45dd9711446916b179bc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:08.883547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:08.883832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:08.884112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:08.900078 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:08.901289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:08.904817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e30bbe21af428faa5ec0eab414abf5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.905549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e30bbe21af428faa5ec0eab414abf5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.906173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e30bbe21af428faa5ec0eab414abf5'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:08.925781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1274/2552] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:08 2022] DELETE /identity/v3/users/06e30bbe21af428faa5ec0eab414abf5 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:08.963756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1279/2553] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:08.973862 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:08.976172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.040162 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447d0f-d255-4267-9bb7-a4f9154429e4 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.043875 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.044551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.044863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.045140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.098886 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:09.112324 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.113860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0363-366b-4d85-92ff-27d9189cd1a0 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.123344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1275/2554] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:09.198720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.200226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.219198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f97626-28cf-4d77-88b6-fb2b79ca96d8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.226119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.227269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.227560 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.227879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] PATH_INFO: `/v3/projects/61e2a8b0891548428759a52462a2feaa` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.239806 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e2a8b0891548428759a52462a2feaa)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.242450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.336803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e2a8b0891548428759a52462a2feaa', 'request_id': 'req-9ce73b84-247a-4270-b9dd-dded3650b43f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.343110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e2a8b0891548428759a52462a2feaa', 'request_id': 'req-9ce73b84-247a-4270-b9dd-dded3650b43f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.356108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e2a8b0891548428759a52462a2feaa', 'request_id': 'req-9ce73b84-247a-4270-b9dd-dded3650b43f'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.430084 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] The token cache is being invalidate because project 61e2a8b0891548428759a52462a2feaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:09.430376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.431003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-9ce73b84-247a-4270-b9dd-dded3650b43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.432647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1280/2555] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:09 2022] DELETE /identity/v3/projects/61e2a8b0891548428759a52462a2feaa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:09.437560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.437935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.438271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.455813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:09.457138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.515588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1276/2556] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:09.523902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.524194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.524471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.545836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:09.545836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.547100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.548509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.554209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1281/2557] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:09.558881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.562011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02accf-1da8-4413-b4ec-1b73eed4b4bd None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.568155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.568499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf411698-6210-4845-9c70-f73aa9cda026 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.572204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.572204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.572204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] PATH_INFO: `/v3/projects/cb846b3a2be74349bdfdb9d3fdd3ed61` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.572204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb846b3a2be74349bdfdb9d3fdd3ed61)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.572970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.575878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.576674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.577053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.577546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.578429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:09.579049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.581251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.596542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1282/2558] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:09.602304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.612020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75285ba-ca9e-4b1c-bb1b-e09d26ed830f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.616956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.617639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.617923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.618201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.618886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.620886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.634999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:09.696496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb846b3a2be74349bdfdb9d3fdd3ed61', 'request_id': 'req-c93c531e-0892-492e-96b0-5edc3a133991'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.703181 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb846b3a2be74349bdfdb9d3fdd3ed61', 'request_id': 'req-c93c531e-0892-492e-96b0-5edc3a133991'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.710220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb846b3a2be74349bdfdb9d3fdd3ed61', 'request_id': 'req-c93c531e-0892-492e-96b0-5edc3a133991'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.721362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1283/2559] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:09 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:09.726740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.789227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967e27-d01e-4e05-8997-36bf00ed1f29 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.794472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.795159 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.795449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.804094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.804094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.804094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.825410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] The token cache is being invalidate because project cb846b3a2be74349bdfdb9d3fdd3ed61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:09.825703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.826353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:09.827947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1277/2560] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:09 2022] DELETE /identity/v3/projects/cb846b3a2be74349bdfdb9d3fdd3ed61 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:09.831850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.833195 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.898396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c531e-0892-492e-96b0-5edc3a133991 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.903102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1284/2561] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:09 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:09.904400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.905827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.906012 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:09.906700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.906983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.907299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] PATH_INFO: `/v3/users/524d8335c8a34a99aa3108091ad79184` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.908520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=524d8335c8a34a99aa3108091ad79184)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.913171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.918506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dea93-7265-481d-a32f-d54f9c91b904 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.923355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.924008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.924301 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.924599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.924728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:09.925338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.926971 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.937582 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1285/2562] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:09.941842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.951765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941625-3617-4a34-8671-00d855674f20 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.956469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.957126 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.957456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.957720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:09.958431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:09.960607 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:09.971534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1286/2563] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:09.976246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:09.985816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc66ee-391e-4075-b81f-06d55662c653 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:09.990612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:09.991449 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:09.991735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:09.992013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] PATH_INFO: `/v3/projects/c72d6318734e45e8890bed7246ac8e92/users/a9ec7c5401274c41a1c3a34632f60071/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:10.007181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72d6318734e45e8890bed7246ac8e92, user_id=a9ec7c5401274c41a1c3a34632f60071, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:10.010148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb825-9a20-4e9c-a921-478a395c5c21 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:10.105618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1287/2564] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:09 2022] PUT /identity/v3/projects/c72d6318734e45e8890bed7246ac8e92/users/a9ec7c5401274c41a1c3a34632f60071/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:10.117162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:10.117458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:10.117735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:10.143179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524d8335c8a34a99aa3108091ad79184'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:10.145685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] MFA Rules not processed for user `a9ec7c5401274c41a1c3a34632f60071`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:10.146952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:10.150912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524d8335c8a34a99aa3108091ad79184'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:10.211740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1288/2565] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:10.219140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524d8335c8a34a99aa3108091ad79184'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:10.222894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524d8335c8a34a99aa3108091ad79184'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:10.222894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524d8335c8a34a99aa3108091ad79184'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:10.225967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:10.227390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:10.244904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1278/2566] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:09 2022] DELETE /identity/v3/users/524d8335c8a34a99aa3108091ad79184 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:10.265992 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:10.314678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185240f1-80fe-432d-951d-12b50b158cc9 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:10.383464 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:10.384127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:10.384409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:10.384716 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:10.386707 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:10.450007 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:10.451742 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:10.463072 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1289/2567] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:10.675783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:10.676892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:10.701500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:10.706461 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:10.756051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9a588-b668-45b1-ae71-953b2664b9de admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:10.760874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:10.761779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:10.762061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:10.762338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] PATH_INFO: `/v3/users/a9ec7c5401274c41a1c3a34632f60071` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:10.763584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ec7c5401274c41a1c3a34632f60071)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:10.765586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:10.785999 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bdf49-c54f-480b-8c5b-f4fdc86ca05b service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:10.791249 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:10.792136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:10.792453 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:10.792762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] PATH_INFO: `/v3/projects/e0646706687a45fc95eb70c6deadd1a2` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:10.793946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0646706687a45fc95eb70c6deadd1a2)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:10.795947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:10.803142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:11.005017 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ec7c5401274c41a1c3a34632f60071'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.006542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0646706687a45fc95eb70c6deadd1a2', 'request_id': 'req-916bab47-fdd5-4ac4-bf50-e866c982a785'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.012197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ec7c5401274c41a1c3a34632f60071'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.013629 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0646706687a45fc95eb70c6deadd1a2', 'request_id': 'req-916bab47-fdd5-4ac4-bf50-e866c982a785'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.020613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0646706687a45fc95eb70c6deadd1a2', 'request_id': 'req-916bab47-fdd5-4ac4-bf50-e866c982a785'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.122049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ec7c5401274c41a1c3a34632f60071'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.122770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ec7c5401274c41a1c3a34632f60071'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.123465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cddc14cc-657c-44d7-a475-f8c5f418c6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ec7c5401274c41a1c3a34632f60071'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.141675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1279/2568] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:10 2022] DELETE /identity/v3/users/a9ec7c5401274c41a1c3a34632f60071 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:11.146504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.147354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.147354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.165696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:11.167142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.171379 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:11.207379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] The token cache is being invalidate because project e0646706687a45fc95eb70c6deadd1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:11.207686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.208297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-916bab47-fdd5-4ac4-bf50-e866c982a785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:11.210441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1290/2569] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:10 2022] DELETE /identity/v3/projects/e0646706687a45fc95eb70c6deadd1a2 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:11.215170 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204cf8-aa79-4b22-a74f-3af6ab82ab3c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.215760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204cf8-aa79-4b22-a74f-3af6ab82ab3c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.215760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204cf8-aa79-4b22-a74f-3af6ab82ab3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.229323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1280/2570] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:11.234574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e6204cf8-aa79-4b22-a74f-3af6ab82ab3c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:11.235779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e6204cf8-aa79-4b22-a74f-3af6ab82ab3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.243645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.249902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1291/2571] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:11.250008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.251540 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.251829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.252109 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.272677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:11.272677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.280099 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1292/2572] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:11.284172 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.293236 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724dcb6-bc73-4e80-a343-047478c197db None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.297958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.298593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.298872 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.299140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.299846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:11.300147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.301761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.302203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffb882-1dc1-4f8c-9b6e-07f92ae10577 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.305876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.306521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.306805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.307083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.316706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1293/2573] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:39:11.316855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.317642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.318628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.327848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be268a5-40b7-4a1c-8cb2-6c20ce58802c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.328321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1281/2574] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:11.332504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.333314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.334210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.334486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.335165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.337089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.462578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1294/2575] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:11.465272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.475520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd3b6-156c-4642-9de1-a26747f5ea8c service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.531929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.532510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.532820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.533093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.533779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.535733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.628468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1282/2576] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:11.630051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.641886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb10182-da2f-41a4-92ae-72ebb7976283 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.647028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.647632 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.647980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.648341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.649226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.650942 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.662659 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1295/2577] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:11.666690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.678339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efd4b-e575-4bd8-936b-8103bb27dd2d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.683166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.683898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.684185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.684461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.685230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.686974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.698350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1283/2578] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:11.702813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.712908 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f621-b3b2-4139-8b17-81c78eca6158 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.717840 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.720156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.720156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.720156 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] PATH_INFO: `/v3/projects/bce37fdcaecf42dfb845b3ad97db1e95/users/c52a165644d4494db5c4d8fe6ad19c19/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.734246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce37fdcaecf42dfb845b3ad97db1e95, user_id=c52a165644d4494db5c4d8fe6ad19c19, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.736923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7bf03-a977-4b9d-87cd-53c052aad979 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.815718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.818467 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.835910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118c054-6ad4-48d9-a137-8e9861fab719 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:11.846508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1296/2579] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:11 2022] PUT /identity/v3/projects/bce37fdcaecf42dfb845b3ad97db1e95/users/c52a165644d4494db5c4d8fe6ad19c19/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:11.854358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.854652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.854936 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.869965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:11.882273 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] MFA Rules not processed for user `c52a165644d4494db5c4d8fe6ad19c19`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:11.883500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:11.899614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:11.900509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:11.903439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:11.903439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] PATH_INFO: `/v3/projects/c72d6318734e45e8890bed7246ac8e92` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:11.903439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72d6318734e45e8890bed7246ac8e92)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:11.904312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:11.946927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:11.950686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1297/2580] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:11.963796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:11.965187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.023096 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33271ec4-0a32-4dbd-9599-e3dd2781d11c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.025825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72d6318734e45e8890bed7246ac8e92', 'request_id': 'req-3682a921-d34f-4147-95b3-3a95274230f0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.032154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72d6318734e45e8890bed7246ac8e92', 'request_id': 'req-3682a921-d34f-4147-95b3-3a95274230f0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.038273 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72d6318734e45e8890bed7246ac8e92', 'request_id': 'req-3682a921-d34f-4147-95b3-3a95274230f0'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.074135 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.074804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.075030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.075307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.126857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] The token cache is being invalidate because project c72d6318734e45e8890bed7246ac8e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:12.127145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.127798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3682a921-d34f-4147-95b3-3a95274230f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.129417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1284/2581] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:11 2022] DELETE /identity/v3/projects/c72d6318734e45e8890bed7246ac8e92 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:12.134820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.135086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d645-2a38-4280-83be-e84dab59c649 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.135410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d645-2a38-4280-83be-e84dab59c649 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.135686 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d645-2a38-4280-83be-e84dab59c649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.136539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.146306 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1298/2582] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:12.153780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-c0e8d645-2a38-4280-83be-e84dab59c649 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:12.157922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c0e8d645-2a38-4280-83be-e84dab59c649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.165705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.233487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1285/2583] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:12.234080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529875e3-3a36-4adb-840d-230df4229d60 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.235255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.235542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.235965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.238285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.239331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.240425 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.240882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] PATH_INFO: `/v3/users/c52a165644d4494db5c4d8fe6ad19c19` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.242290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52a165644d4494db5c4d8fe6ad19c19)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.244605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.254807 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:12.257366 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.267508 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1286/2584] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:12.269310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.278552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7f2a-c5c4-4b3d-bd59-ba358cd113f8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.285814 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.287033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.287435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.287711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.288476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:12.288838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.291774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.303172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1287/2585] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:12.307744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.317566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93255-4356-49ec-8d32-74a16d782dd0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.323292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.326242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.326242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.326242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.326242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.327121 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.517812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1288/2586] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:12.522357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.543313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:12.564176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52a165644d4494db5c4d8fe6ad19c19'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.571015 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52a165644d4494db5c4d8fe6ad19c19'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.571683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52a165644d4494db5c4d8fe6ad19c19'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.575784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:12.587318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d8f38-f302-42b3-bf3e-4da86ba97fae admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.591952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.592644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.592926 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.593200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.593893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.595826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.664939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52a165644d4494db5c4d8fe6ad19c19'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.665584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c1cc33d4-954c-4c4a-996f-7c86a5ea8ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52a165644d4494db5c4d8fe6ad19c19'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:12.684531 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1299/2587] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:12 2022] DELETE /identity/v3/users/c52a165644d4494db5c4d8fe6ad19c19 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:12.689042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.689328 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.689598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.706118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:12.707347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.731931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1289/2588] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:12.733068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.734666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.746519 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1933-c2a7-4477-b199-b6318b955ef7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.772232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1300/2589] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:12.781919 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.783283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:12.805717 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.806512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.806798 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.807726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.808475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.810123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.820598 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1290/2590] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:12.824511 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.836206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09057a6-4039-4d80-b09d-ba562511c7c8 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.843300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.843300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.843300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.843300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.843712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.844596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.846204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273c497-c734-4a83-b8b3-da39c5f91edb None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.850133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.850763 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.851045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.851321 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.854955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1291/2591] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:12.858989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:12.862071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.863622 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd747e-0ff5-4021-be8b-5014912c5082 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.868824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4365b-61e9-43da-acfc-83f29a1bc54b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:12.873563 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:12.873818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1301/2592] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:12.874471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.874863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.875062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] PATH_INFO: `/v3/projects/924f6340c17d4ab3a4b86c6e20f8542b/users/4845620e681d4821b36d4d7263736307/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:12.890923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924f6340c17d4ab3a4b86c6e20f8542b, user_id=4845620e681d4821b36d4d7263736307, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:12.893943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:12.985203 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1292/2593] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:12 2022] PUT /identity/v3/projects/924f6340c17d4ab3a4b86c6e20f8542b/users/4845620e681d4821b36d4d7263736307/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:12.995600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:12.995881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:12.996155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.024544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] MFA Rules not processed for user `4845620e681d4821b36d4d7263736307`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:13.028503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.046888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:13.094198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1302/2594] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:13.098672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.172301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c974-09f1-4607-bdc1-864b1556d503 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.177656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.178436 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.178725 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.179003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.179722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.181724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.197038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:13.272259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1293/2595] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:13.276305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.287313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a734-8614-4f1d-ba14-8b4b2e687ce0 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.300932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.304242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.357608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.358241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.358528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.359101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.359512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.361494 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.373841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c42fa-200a-4901-b01f-a2257e9bd092 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.378703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.379818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.380921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.381520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] PATH_INFO: `/v3/projects/bce37fdcaecf42dfb845b3ad97db1e95` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.382701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce37fdcaecf42dfb845b3ad97db1e95)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.384644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.454288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1303/2596] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:13.457705 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.467280 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5140-ebbe-43eb-8fac-9387760fe2d2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.471895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.472533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.472851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.473134 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.473853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.476199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.489161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1304/2597] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:13.490874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.490997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce37fdcaecf42dfb845b3ad97db1e95', 'request_id': 'req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:13.497711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce37fdcaecf42dfb845b3ad97db1e95', 'request_id': 'req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:13.500404 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4763a1-ab72-4e49-b96c-e14fb0f98703 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.503385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce37fdcaecf42dfb845b3ad97db1e95', 'request_id': 'req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:13.556945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.557830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.558116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.558390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] PATH_INFO: `/v3/projects/53e90132a491410cafff87c3114fe516/users/16d1cbfde14a48259bc162a80797bc06/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.572451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e90132a491410cafff87c3114fe516, user_id=16d1cbfde14a48259bc162a80797bc06, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.575215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02b1d-56d5-4aa1-8d6a-69ef03ddb2f6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.613055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] The token cache is being invalidate because project bce37fdcaecf42dfb845b3ad97db1e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:13.613280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:13.613944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-cfdbeff0-cf15-4e5f-a6b5-56d2982cdbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:13.616152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1294/2598] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:13 2022] DELETE /identity/v3/projects/bce37fdcaecf42dfb845b3ad97db1e95 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:13.621036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.621323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.621599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.640067 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:13.641275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.649401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1295/2599] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:13.687484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1305/2600] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:13 2022] PUT /identity/v3/projects/53e90132a491410cafff87c3114fe516/users/16d1cbfde14a48259bc162a80797bc06/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:13.692019 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.693520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.706482 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c6e19-7e6e-45ca-a19f-1a0e72f3c9d2 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.713704 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.713982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.714289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.737556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:13.738880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.761627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:13.785690 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.787208 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.787327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.787572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.788309 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.789969 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.804056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1296/2601] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:13.805532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.812286 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1306/2602] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:13.815427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d3952-da51-4665-9df5-ded01dcc9842 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.816700 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.820331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.821254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.821533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.821805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] PATH_INFO: `/v3/projects/53e90132a491410cafff87c3114fe516/users/16d1cbfde14a48259bc162a80797bc06/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.823764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e90132a491410cafff87c3114fe516, user_id=16d1cbfde14a48259bc162a80797bc06, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.826440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928024-971e-4062-bc7a-f7c9fde2bceb admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.828662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff8277-a7a6-4bbd-b85c-aede38cc592e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.831527 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.832179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.832469 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.832783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.834035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:13.834405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.836011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.837480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1297/2603] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:13 2022] PUT /identity/v3/projects/53e90132a491410cafff87c3114fe516/users/16d1cbfde14a48259bc162a80797bc06/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:13.848750 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.850461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.850461 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.850663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1307/2604] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:39:13.854777 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.856098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:13.866209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe187-95ff-46bb-a658-18698a22865e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:13.877344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] MFA Rules not processed for user `16d1cbfde14a48259bc162a80797bc06`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:13.878576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:13.934149 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:13.935121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:13.935424 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:13.935696 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:13.936567 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:13.938808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:13.960164 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1298/2605] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:13.971311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:13.972676 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.030107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323fe8-d034-4d65-87e6-2c8d48fb5849 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.033753 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.034423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.034711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.034988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.044844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.046272 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.055706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1299/2606] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:14.070551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1308/2607] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:13 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:14.071613 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.081743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2a2fc-b463-4007-bc94-78e4d3e40794 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.121008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.122720 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.142300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.142970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.143255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.143531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.144220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.146769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.189290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae722b-fd67-46f0-a664-2ee3b72f94a1 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.193985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.195035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.195246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.195574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] PATH_INFO: `/v3/users/8d1d999bc7454469ba7e6afe8d6092e9` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.196957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1d999bc7454469ba7e6afe8d6092e9)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.198889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.228382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1300/2608] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:14 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:14.232278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.242409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02018ea1-4a1e-49d0-96dd-c949da671f9e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.247302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.247952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.248236 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.248513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.249279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.250858 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.261223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1301/2609] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:14.262127 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.263861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.306607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:14.321653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816298-656b-4d3a-a3c7-78f7f5c355a6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.325412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.326052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.326383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.326667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.328504 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.385545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.386888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.392906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:14.397315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1302/2610] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:14.398368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.408562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1d999bc7454469ba7e6afe8d6092e9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:14.409269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d843-3657-4775-ad72-0c116a161ea3 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.415651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.416312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.416612 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.416907 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1d999bc7454469ba7e6afe8d6092e9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:14.417025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.417843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.417972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1d999bc7454469ba7e6afe8d6092e9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:14.419698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.431184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1303/2611] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:14.439973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.445637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505955-f87d-4e77-8cb9-d8ea5b9d8214 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.450414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.451286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.451561 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.451947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] PATH_INFO: `/v3/projects/cf690ad6c5694ad0a51944572935e117/users/207d444f601a4dec9c6321d88288bc80/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.467547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf690ad6c5694ad0a51944572935e117, user_id=207d444f601a4dec9c6321d88288bc80, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.470213 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65709cb-07ed-4a3e-8396-6e9bd0de10de admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.493398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1d999bc7454469ba7e6afe8d6092e9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:14.494040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ca6a3f22-d2ca-46cb-876a-e13abb18395e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1d999bc7454469ba7e6afe8d6092e9'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:14.512214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1309/2612] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:14 2022] DELETE /identity/v3/users/8d1d999bc7454469ba7e6afe8d6092e9 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:14.516676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.516969 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.517248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.535119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:14.536341 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.579735 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1304/2613] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:14 2022] PUT /identity/v3/projects/cf690ad6c5694ad0a51944572935e117/users/207d444f601a4dec9c6321d88288bc80/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:14.590728 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.591124 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.591401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.594612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1310/2614] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:14.605952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.607501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.620719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] MFA Rules not processed for user `207d444f601a4dec9c6321d88288bc80`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:14.621948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.668276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f269e-f5bf-4eef-b00a-25ad16182672 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.686287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1305/2615] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:14.698640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:14.699655 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:14.720453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:14.721613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.722472 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.722593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.722809 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.753930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:14.758713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a509fb-6825-45a2-acea-63d3b4297912 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:14.762442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:14.763087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:14.763370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:14.763643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:14.772145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.773634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.782730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1306/2616] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:14.791042 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:14.792689 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:14.812049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1311/2617] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:15.335843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:15.338075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:15.363155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af57063-78c8-4506-acf1-3d83d9212f98 service glance] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:15.367809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:15.368727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:15.369016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:15.369310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] PATH_INFO: `/v3/projects/8af4ee94483548de918820841c0b1546` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:15.370599 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af4ee94483548de918820841c0b1546)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:15.372707 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:15.484054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af4ee94483548de918820841c0b1546', 'request_id': 'req-d51618dd-4e97-4373-bb69-2f76b66738e9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.490491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af4ee94483548de918820841c0b1546', 'request_id': 'req-d51618dd-4e97-4373-bb69-2f76b66738e9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.496197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af4ee94483548de918820841c0b1546', 'request_id': 'req-d51618dd-4e97-4373-bb69-2f76b66738e9'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.607397 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] The token cache is being invalidate because project 8af4ee94483548de918820841c0b1546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:15.607693 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.608367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.609958 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1307/2618] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:15 2022] DELETE /identity/v3/projects/8af4ee94483548de918820841c0b1546 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:15.614022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:15.624198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007e71d-b2ad-4b54-a393-3157b5dbea23 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:15.700966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:15.701847 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:15.702130 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:15.702483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] PATH_INFO: `/v3/users/85de6c71742f4513ab48bd5e5f09c373` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:15.703727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85de6c71742f4513ab48bd5e5f09c373)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:15.705709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:15.947609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85de6c71742f4513ab48bd5e5f09c373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.954238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85de6c71742f4513ab48bd5e5f09c373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:15.954888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85de6c71742f4513ab48bd5e5f09c373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.047650 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85de6c71742f4513ab48bd5e5f09c373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.048250 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-60e12919-5185-49c6-a746-4d241a0da0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85de6c71742f4513ab48bd5e5f09c373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.066069 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1312/2619] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:15 2022] DELETE /identity/v3/users/85de6c71742f4513ab48bd5e5f09c373 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:16.558933 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:16.560446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:16.574653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51618dd-4e97-4373-bb69-2f76b66738e9 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:16.634960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:16.635848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:16.636125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:16.636396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] PATH_INFO: `/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:16.637604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1a49af6d9247f59ee6e5d83c2bfe3e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:16.639488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:16.789986 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:16.793108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1a49af6d9247f59ee6e5d83c2bfe3e', 'request_id': 'req-1bac48ed-c21b-4300-ab25-9a12ba3af9af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.799513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1a49af6d9247f59ee6e5d83c2bfe3e', 'request_id': 'req-1bac48ed-c21b-4300-ab25-9a12ba3af9af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.805517 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1a49af6d9247f59ee6e5d83c2bfe3e', 'request_id': 'req-1bac48ed-c21b-4300-ab25-9a12ba3af9af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.921471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] The token cache is being invalidate because project 0b1a49af6d9247f59ee6e5d83c2bfe3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:16.921791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.922420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1bac48ed-c21b-4300-ab25-9a12ba3af9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:16.923971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1308/2620] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:16 2022] DELETE /identity/v3/projects/0b1a49af6d9247f59ee6e5d83c2bfe3e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:16.929108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:16.929601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:16.929818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:16.948702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:16.950034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:16.991439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:17.012016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1313/2621] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:17.020434 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.021258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.021539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.040465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:17.041710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:17.051513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1309/2622] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:17.054181 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.064082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8100-6fd2-40cd-b5d4-0fe3af0b2064 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.068825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.069488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.069776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.070055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.070785 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:17.071095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.072743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.085333 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1314/2623] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:17.089730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.100845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75158a-da5e-4b84-9a05-76340133a8cd None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.105786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.106465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.106747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.107021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.107724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.109651 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.273699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1310/2624] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/projects => generated 391 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:17.277683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.292572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5793-fc5b-476c-867a-46fccd34d0bf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.352485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.353192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.353471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.353743 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.354445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.356380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.460238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1315/2625] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:17.464289 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.475490 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ef67e-8fce-4e82-a310-1aa235f3981b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.480080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.480772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.481051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.485841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.485841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.485841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.494154 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1311/2626] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:17.498379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.509335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d6198-b36f-4757-b52b-edc95f01af0b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.514148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.514834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.515116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.515389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.516161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.517869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7dd4-95de-47fc-9f55-7703d8a28ea7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.529257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1316/2627] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:17.533491 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.544070 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eb08-a0c3-4601-80af-89997bbe7b54 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.549207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.550115 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.550399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.550674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] PATH_INFO: `/v3/projects/1e042d0683e742fd8c4da3a229107fa8/users/fbd78fe5fa0545c8a0bed7bca3bf1b24/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.567021 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e042d0683e742fd8c4da3a229107fa8, user_id=fbd78fe5fa0545c8a0bed7bca3bf1b24, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.569760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.718126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1312/2628] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:17 2022] PUT /identity/v3/projects/1e042d0683e742fd8c4da3a229107fa8/users/fbd78fe5fa0545c8a0bed7bca3bf1b24/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:17.730484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.730770 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.731049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.759801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] MFA Rules not processed for user `fbd78fe5fa0545c8a0bed7bca3bf1b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:17.761164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:17.826656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1317/2629] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:17.834007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:17.899144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08548a-ae79-441f-be82-4c59bb50a2cf admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:17.903721 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:17.904377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:17.904699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:17.904972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:17.905703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:17.907583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:17.993371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1313/2630] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:17.997501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.010576 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859989e0-65fc-400e-8894-2435f02ee265 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.044564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:18.068420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.069125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.069610 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.070054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.071142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.073167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.214233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1318/2631] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:17 2022] POST /identity/v3/users => generated 303 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:18.218458 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.229911 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00a76-a85a-4f5a-9be6-9347fc770145 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.234575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.235645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.235922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.237350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.237350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.238688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.250024 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1314/2632] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:18.264307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.265073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7754ef-eddb-47e9-b62c-625a6198dafb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.272022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.272022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.272022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.272022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] PATH_INFO: `/v3/projects/1b10e203222940e294d194de6d5313c0/users/b0cb0b89dd444b9b8f4b829a3f378947/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.287117 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b10e203222940e294d194de6d5313c0, user_id=b0cb0b89dd444b9b8f4b829a3f378947, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.289863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.362059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1319/2633] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:18 2022] PUT /identity/v3/projects/1b10e203222940e294d194de6d5313c0/users/b0cb0b89dd444b9b8f4b829a3f378947/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:18.366048 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.377455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25745cfd-7d8b-4a6f-b691-46d67556a899 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.421476 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:18.436158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.436953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.437126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.437403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.438138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.439731 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775bfac-4a93-438b-a088-5aa7352132e2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.450197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1315/2634] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:18.454300 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.465175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdabb-5c7d-46a9-9dc5-f25b1e2f41ff admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.469786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.470664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.470946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.471223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] PATH_INFO: `/v3/projects/1b10e203222940e294d194de6d5313c0/users/b0cb0b89dd444b9b8f4b829a3f378947/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.473223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b10e203222940e294d194de6d5313c0, user_id=b0cb0b89dd444b9b8f4b829a3f378947, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.476731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.488233 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1320/2635] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:18 2022] PUT /identity/v3/projects/1b10e203222940e294d194de6d5313c0/users/b0cb0b89dd444b9b8f4b829a3f378947/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:18.498710 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.499000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.499283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.526361 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] MFA Rules not processed for user `b0cb0b89dd444b9b8f4b829a3f378947`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:18.527775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:18.597984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1316/2636] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:18.611013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.612405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:18.671838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246262c8-9745-4521-9bf5-ac623bd229f3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.675562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.676207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.676490 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.676798 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.685870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.687297 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.697354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1321/2637] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:18.749791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:18.759938 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e3475-d36c-4a2d-ac1c-190ba780e876 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:18.764663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:18.765313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:18.765621 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:18.765897 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:18.826061 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:18.827529 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e64a5b-c93d-487b-9b6c-1a8430b9b18d service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:18.837226 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1317/2638] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:19.744252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:19.745721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:19.813958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427e075-d452-4d99-8efc-ad6bdd6688fd service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:19.819029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:19.825071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:19.825071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:19.825071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] PATH_INFO: `/v3/users/4845620e681d4821b36d4d7263736307` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:19.825071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4845620e681d4821b36d4d7263736307)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:19.825071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:20.046517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:20.056354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4845620e681d4821b36d4d7263736307'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.063853 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4845620e681d4821b36d4d7263736307'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.064559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4845620e681d4821b36d4d7263736307'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.167782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4845620e681d4821b36d4d7263736307'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.168411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4845620e681d4821b36d4d7263736307'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.188185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1322/2639] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:19 2022] DELETE /identity/v3/users/4845620e681d4821b36d4d7263736307 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:20.193286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:20.193790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:20.193921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:20.210913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:20.212157 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:20.274368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1318/2640] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:20.284095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:20.285538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:20.342824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d045-daac-4480-b74a-a1710a42248c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:20.346498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:20.347144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:20.347431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:20.347712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:20.357359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:20.358803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:20.369495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1323/2641] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:20.821333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:20.822764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:20.831810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:20.837064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc44039-1afd-4e47-8574-06c941a29d18 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:20.841335 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:20.842207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:20.842496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:20.842774 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] PATH_INFO: `/v3/projects/924f6340c17d4ab3a4b86c6e20f8542b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:20.843967 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=924f6340c17d4ab3a4b86c6e20f8542b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:20.846007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:20.980305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924f6340c17d4ab3a4b86c6e20f8542b', 'request_id': 'req-5dde7bf6-056b-497d-beff-1a1aca047afc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.986869 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924f6340c17d4ab3a4b86c6e20f8542b', 'request_id': 'req-5dde7bf6-056b-497d-beff-1a1aca047afc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:20.993359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924f6340c17d4ab3a4b86c6e20f8542b', 'request_id': 'req-5dde7bf6-056b-497d-beff-1a1aca047afc'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.088116 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] The token cache is being invalidate because project 924f6340c17d4ab3a4b86c6e20f8542b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:21.088412 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.092849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.092849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1319/2642] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:20 2022] DELETE /identity/v3/projects/924f6340c17d4ab3a4b86c6e20f8542b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:21.094930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:21.105313 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5680b7d-b96f-425e-b046-5c6a07a57884 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:21.168636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:21.169538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:21.169816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:21.170090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] PATH_INFO: `/v3/users/16d1cbfde14a48259bc162a80797bc06` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:21.171340 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d1cbfde14a48259bc162a80797bc06)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:21.173391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:21.393556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d1cbfde14a48259bc162a80797bc06'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.400487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d1cbfde14a48259bc162a80797bc06'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.401187 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d1cbfde14a48259bc162a80797bc06'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.402885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:21.458822 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d1cbfde14a48259bc162a80797bc06'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.459542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3a77ecd-e91c-4e73-94f3-5f3997291b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d1cbfde14a48259bc162a80797bc06'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:21.477586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1324/2643] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:21 2022] DELETE /identity/v3/users/16d1cbfde14a48259bc162a80797bc06 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:21.940989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:21.942737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:21.962855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7bf6-056b-497d-beff-1a1aca047afc admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.032832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.033694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.033974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.034247 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] PATH_INFO: `/v3/projects/53e90132a491410cafff87c3114fe516` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.035452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e90132a491410cafff87c3114fe516)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.037943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.176260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e90132a491410cafff87c3114fe516', 'request_id': 'req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:22.183418 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e90132a491410cafff87c3114fe516', 'request_id': 'req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:22.189786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e90132a491410cafff87c3114fe516', 'request_id': 'req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:22.293850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] The token cache is being invalidate because project 53e90132a491410cafff87c3114fe516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:22.294141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:22.294772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-2aad729d-30b7-4f80-ba2e-3e95e44d9e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:22.297065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1320/2644] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:21 2022] DELETE /identity/v3/projects/53e90132a491410cafff87c3114fe516 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:22.301961 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.302423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.302545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.319823 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:22.321077 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:22.384290 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1325/2645] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:22.391944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.392183 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.392363 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.413781 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:22.414974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:22.422827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1321/2646] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:22.427113 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.437150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c418-3a16-4439-8052-e0b3224dd61f None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.446281 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.446957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.452989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.452989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.452989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:22.452989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.452989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.466150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1326/2647] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:22.467230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.476887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc378efc-dbd8-4b30-88b8-77bf586da825 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.481724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.482378 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.482658 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.483007 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.483637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.485709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.494558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:22.621431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1322/2648] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:22 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:22.625669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.635782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8955-8c20-409d-9d3a-aa3912871b2f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.698559 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.699267 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.699554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.699816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.700525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.702594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.819681 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1327/2649] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:22 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:22.823739 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.835827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b421935-280c-4a15-b17d-9f4bc092b90c admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.840551 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.841243 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.841547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.841820 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.842547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.844130 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.858649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1323/2650] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:22.871526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.882065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58f2bc-c54d-47c5-a783-81490a941196 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.887722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.888456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.888768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.889044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.889774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.891485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b646e40-b261-46e8-97c3-fec2e2f8c937 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:22.903086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1328/2651] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:22.907329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:22.918315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1eb2-f7f4-42d5-a08e-3004db1bd619 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:22.923564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:22.924468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:22.924779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:22.925052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] PATH_INFO: `/v3/projects/d2c28709bc144ced87dee60a0bc1f6d7/users/5e4425edd14a4a2dbb27a9b097350502/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:22.940662 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c28709bc144ced87dee60a0bc1f6d7, user_id=5e4425edd14a4a2dbb27a9b097350502, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:22.943328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.043216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1324/2652] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:22 2022] PUT /identity/v3/projects/d2c28709bc144ced87dee60a0bc1f6d7/users/5e4425edd14a4a2dbb27a9b097350502/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:23.053722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.054010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.054284 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.082525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] MFA Rules not processed for user `5e4425edd14a4a2dbb27a9b097350502`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:23.084312 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:23.114400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:23.151972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1329/2653] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:23.156780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.229789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f5a8-3066-4009-9436-6def55d03a3f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.234873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.235534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.235846 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.236122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.236872 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.238817 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.324498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1325/2654] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:23 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:23.328138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.339429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c007b-5a4f-47d0-81d9-d8ce61648879 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.396217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.396922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.397209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.397483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.398169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.400092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.539959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1330/2655] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:23 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:23.544414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.555630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d8b2f-7e45-4717-864a-d19b523c7831 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.560196 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.560891 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.561174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.561448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.562174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.563764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.574303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1326/2656] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:23.578512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.589086 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ad5b-b3dc-4334-9b30-8696e8eae982 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.593678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.595215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.595215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.595215 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] PATH_INFO: `/v3/projects/ac368b03325e4eecade35db48402dc21/users/17cf7e0210e641349264389d6430d31c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.610263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac368b03325e4eecade35db48402dc21, user_id=17cf7e0210e641349264389d6430d31c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.612953 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.699456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1331/2657] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:23 2022] PUT /identity/v3/projects/ac368b03325e4eecade35db48402dc21/users/17cf7e0210e641349264389d6430d31c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:23.703326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.714424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7807-d49f-43c8-a7f0-f0f788b0ef89 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.774987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.775689 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.776123 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.776407 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.777170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.778819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95221f11-ccc8-41dd-9413-bc67ffbe89a1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.790898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1327/2658] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:23.794876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.805435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be791245-cbb1-40d8-813a-5c3f2c742ddb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:23.810131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:23.811178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.811456 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.811726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] PATH_INFO: `/v3/projects/ac368b03325e4eecade35db48402dc21/users/17cf7e0210e641349264389d6430d31c/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.813712 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac368b03325e4eecade35db48402dc21, user_id=17cf7e0210e641349264389d6430d31c, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:23.816369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:23.826360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1332/2659] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:23 2022] PUT /identity/v3/projects/ac368b03325e4eecade35db48402dc21/users/17cf7e0210e641349264389d6430d31c/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:23.838666 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:23.839211 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:23.840604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:23.866824 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] MFA Rules not processed for user `17cf7e0210e641349264389d6430d31c`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:23.868028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:23.880084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:23.937915 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1328/2660] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:23.949955 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:23.951649 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:24.011443 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bf4cf-5fac-42ed-80ab-0ec2dcf81d50 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:24.015223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:24.016098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:24.016430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:24.016744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:24.030224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:24.031693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:24.046751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1333/2661] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:25.064468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.065961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:25.134155 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714a7d4-6a2c-4c09-be32-c3e2b467bd28 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.138769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.139640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.139921 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.140197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] PATH_INFO: `/v3/users/fbd78fe5fa0545c8a0bed7bca3bf1b24` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.141449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd78fe5fa0545c8a0bed7bca3bf1b24)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.143433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.190774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.199994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb06c6-5395-484a-9dfd-69eede8a3cbb service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.204518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17cf7e0210e641349264389d6430d31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac368b03325e4eecade35db48402dc21', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.205192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.205479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.205758 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.206530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:25.206837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.209384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.220975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1334/2662] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:25.225621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.234767 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3d53d-ad43-4c45-9584-b27f9b9959ed tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.239261 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17cf7e0210e641349264389d6430d31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac368b03325e4eecade35db48402dc21', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.239864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.240150 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.240430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.241171 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.243006 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.251683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:25.366427 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1335/2663] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:25 2022] POST /identity/v3/projects => generated 352 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:25.367523 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.371143 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd78fe5fa0545c8a0bed7bca3bf1b24'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.378423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd78fe5fa0545c8a0bed7bca3bf1b24'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.429247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a1ecc-3e01-45c0-94f5-24d18dc92baf tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.433962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17cf7e0210e641349264389d6430d31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac368b03325e4eecade35db48402dc21', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.434639 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.434925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.435227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.435954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:25.436263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.437899 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee86b-bec9-4aca-8e21-208765e1c3c8 tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.449291 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1336/2664] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:39:25.453094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd78fe5fa0545c8a0bed7bca3bf1b24'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.453724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd78fe5fa0545c8a0bed7bca3bf1b24'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.454763 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd78fe5fa0545c8a0bed7bca3bf1b24'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.474488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1329/2665] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:25 2022] DELETE /identity/v3/users/fbd78fe5fa0545c8a0bed7bca3bf1b24 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:25.479000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.479316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.481780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.486687 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.488081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:25.497369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:25.498588 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:25.521892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:25.557442 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1337/2666] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:25.558259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795215ef-1f6e-460b-bcce-e1c424954dee admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.564216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17cf7e0210e641349264389d6430d31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac368b03325e4eecade35db48402dc21', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.565338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.565630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.565924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] PATH_INFO: `/v3/projects/7d1493469c024452bf1fea0ef4c2e5ea` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.567980 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:25.568245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7d1493469c024452bf1fea0ef4c2e5ea)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.569302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:25.570660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.626593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74a7f4-93e2-4cc3-bcb7-ab3a5c9225e3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:25.630391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:25.631055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:25.631336 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:25.631619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:25.640989 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:25.642428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:25.651962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1338/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:25.735900 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1493469c024452bf1fea0ef4c2e5ea', 'request_id': 'req-a72561ec-fa5a-49f9-9967-af1a99bd612b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.743498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1493469c024452bf1fea0ef4c2e5ea', 'request_id': 'req-a72561ec-fa5a-49f9-9967-af1a99bd612b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.749224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1493469c024452bf1fea0ef4c2e5ea', 'request_id': 'req-a72561ec-fa5a-49f9-9967-af1a99bd612b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.853771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] The token cache is being invalidate because project 7d1493469c024452bf1fea0ef4c2e5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:25.854060 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.854709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:25.856319 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1330/2668] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:25 2022] DELETE /identity/v3/projects/7d1493469c024452bf1fea0ef4c2e5ea => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:26.107325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:26.108952 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:26.124046 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ee337-1b69-45a5-b161-1d7e99314895 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:26.150734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:26.182654 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:26.183625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:26.183916 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:26.184196 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] PATH_INFO: `/v3/projects/1e042d0683e742fd8c4da3a229107fa8` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:26.185477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e042d0683e742fd8c4da3a229107fa8)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:26.187438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:26.372619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e042d0683e742fd8c4da3a229107fa8', 'request_id': 'req-039b07c5-49b5-4485-a4c8-a96010627a04'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.378097 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e042d0683e742fd8c4da3a229107fa8', 'request_id': 'req-039b07c5-49b5-4485-a4c8-a96010627a04'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.384663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e042d0683e742fd8c4da3a229107fa8', 'request_id': 'req-039b07c5-49b5-4485-a4c8-a96010627a04'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.521776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] The token cache is being invalidate because project 1e042d0683e742fd8c4da3a229107fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:26.522098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.522729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.524277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1339/2669] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:26 2022] DELETE /identity/v3/projects/1e042d0683e742fd8c4da3a229107fa8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:26.528380 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:26.529906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:26.556398 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72561ec-fa5a-49f9-9967-af1a99bd612b tempest-VolumeQuotaClassesTest-676155355 tempest-VolumeQuotaClassesTest-676155355-project-admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:26.606741 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:26.607532 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:26.607810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:26.608088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] PATH_INFO: `/v3/users/b0cb0b89dd444b9b8f4b829a3f378947` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:26.609284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cb0b89dd444b9b8f4b829a3f378947)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:26.611207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:26.736460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:26.737930 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:26.750508 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b07c5-49b5-4485-a4c8-a96010627a04 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:26.755028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:26.755890 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:26.756165 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:26.756435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] PATH_INFO: `/v3/users/5e4425edd14a4a2dbb27a9b097350502` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:26.757626 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4425edd14a4a2dbb27a9b097350502)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:26.759545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:26.856386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cb0b89dd444b9b8f4b829a3f378947'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.863520 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cb0b89dd444b9b8f4b829a3f378947'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.867311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:26.978825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cb0b89dd444b9b8f4b829a3f378947'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.979712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cb0b89dd444b9b8f4b829a3f378947'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.980339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-5057702f-f109-4bbd-a08d-4b87167b7c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cb0b89dd444b9b8f4b829a3f378947'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:26.993107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4425edd14a4a2dbb27a9b097350502'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.000298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1331/2670] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:26 2022] DELETE /identity/v3/users/b0cb0b89dd444b9b8f4b829a3f378947 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:27.000441 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4425edd14a4a2dbb27a9b097350502'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.000869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4425edd14a4a2dbb27a9b097350502'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.087258 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4425edd14a4a2dbb27a9b097350502'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.087885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4425edd14a4a2dbb27a9b097350502'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.106676 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1340/2671] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:26 2022] DELETE /identity/v3/users/5e4425edd14a4a2dbb27a9b097350502 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:27.111178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.111472 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.111751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.129656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:27.131410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.191723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1332/2672] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:27.217794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:27.219274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.254497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:27.281864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7576-5ce3-4076-aafa-c1c873197b9b admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:27.285594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:27.286255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.286539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.286816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.298951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:27.300418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:27.311188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1341/2673] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:27.490073 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:27.491873 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.509521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689c62-9485-4cd1-8961-3609f7baf0b1 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:27.514395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:27.515417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.515698 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.515972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] PATH_INFO: `/v3/projects/1b10e203222940e294d194de6d5313c0` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.518484 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b10e203222940e294d194de6d5313c0)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:27.520629 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:27.688804 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b10e203222940e294d194de6d5313c0', 'request_id': 'req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.695861 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b10e203222940e294d194de6d5313c0', 'request_id': 'req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.701770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b10e203222940e294d194de6d5313c0', 'request_id': 'req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.740900 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:27.742350 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.755646 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008d6d-d484-4cfa-a793-de5c74bb868a service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:27.813065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:27.813931 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.814214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.814605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] PATH_INFO: `/v3/projects/d2c28709bc144ced87dee60a0bc1f6d7` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.815793 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c28709bc144ced87dee60a0bc1f6d7)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:27.817817 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:27.858200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] The token cache is being invalidate because project 1b10e203222940e294d194de6d5313c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:27.858487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.859108 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-9af4b2fc-cb61-4e0f-ad48-a91c042aeaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.860711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1333/2674] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:27 2022] DELETE /identity/v3/projects/1b10e203222940e294d194de6d5313c0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:27.865963 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d744f-377f-4bf6-9086-aebdb9b7f0aa None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.866253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d744f-377f-4bf6-9086-aebdb9b7f0aa None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.866533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d744f-377f-4bf6-9086-aebdb9b7f0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.885052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-042d744f-377f-4bf6-9086-aebdb9b7f0aa None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:27.886327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-042d744f-377f-4bf6-9086-aebdb9b7f0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.894447 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1334/2675] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:27.903118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.903408 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.903684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.922190 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:27.923373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:27.932136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1335/2676] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:27.936140 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:27.945040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b866-b709-49c0-96d1-ec2b37f9524e None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:27.949927 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:27.950556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:27.950815 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:27.951088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:27.951788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:27.952092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:27.953729 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:27.968466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1336/2677] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:39:27.969711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:27.975644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c28709bc144ced87dee60a0bc1f6d7', 'request_id': 'req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.980135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6529b-0b32-4105-9bec-c30b610a0203 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:27.981554 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c28709bc144ced87dee60a0bc1f6d7', 'request_id': 'req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:27.987947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c28709bc144ced87dee60a0bc1f6d7', 'request_id': 'req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.014125 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:28.040102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.040793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.041071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.041344 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.042047 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.043903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.064103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] The token cache is being invalidate because project d2c28709bc144ced87dee60a0bc1f6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:28.064384 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.065026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.066584 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1342/2678] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:27 2022] DELETE /identity/v3/projects/d2c28709bc144ced87dee60a0bc1f6d7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:28.070377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.071684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:28.084119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0547d-52d2-43f8-8b4f-9a5c0b2e97d8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.088960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.089797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.090075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.090347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] PATH_INFO: `/v3/users/17cf7e0210e641349264389d6430d31c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.091600 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17cf7e0210e641349264389d6430d31c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.093927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.164011 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1337/2679] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:27 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:28.167827 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.169477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:28.233424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded505e-2bc7-47e0-a68b-e5a395f1de5d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.238232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.266812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.266812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.266812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.266812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.266812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.465449 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1338/2680] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:28 2022] POST /identity/v3/users => generated 289 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:28.469312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.479887 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485cd5-3a86-4415-8eb6-70bf57e4affe admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.485743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.486331 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.486615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.486890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.487608 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.489255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.495548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17cf7e0210e641349264389d6430d31c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.499871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1339/2681] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:28.502686 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17cf7e0210e641349264389d6430d31c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.503376 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17cf7e0210e641349264389d6430d31c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.504605 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.505677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:28.514954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac30795-223d-4f1c-bec0-a556a66a9da0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.519446 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.520106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.521028 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.521313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.522022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.523576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.534036 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1340/2682] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:28.538406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.547679 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada0ff-99d1-4dba-9a8e-823ab35f1205 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.552882 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.553704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.559266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.559266 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] PATH_INFO: `/v3/projects/480ccd8e209d40a5963c98c753e6af53/users/aeb638796248423880622b4cc598976e/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.568323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] RBAC: Authorizing `identity:create_grant(project_id=480ccd8e209d40a5963c98c753e6af53, user_id=aeb638796248423880622b4cc598976e, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.570957 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.613090 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17cf7e0210e641349264389d6430d31c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.613721 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0fdf21f5-f3ef-400e-8341-3409b628b209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17cf7e0210e641349264389d6430d31c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:28.631918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1343/2683] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:28 2022] DELETE /identity/v3/users/17cf7e0210e641349264389d6430d31c => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:28.647980 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1341/2684] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:28 2022] PUT /identity/v3/projects/480ccd8e209d40a5963c98c753e6af53/users/aeb638796248423880622b4cc598976e/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:28.658133 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.658416 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.658693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.686998 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] MFA Rules not processed for user `aeb638796248423880622b4cc598976e`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:28.688270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:28.752327 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1344/2685] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:28.764018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:28.765761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:28.823637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e91fb-e32e-465a-8f2d-0aa41678431d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:28.827421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:28.828066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:28.828346 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:28.828649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:28.837918 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:28.839357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7000e35-d645-43b5-b7f7-8f25cd993e7c service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:28.848723 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1342/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:29.083935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:29.085613 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:29.158685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369870de-dd0d-4c1d-a62e-c43c17b9e991 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:29.163271 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:29.164124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.164409 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.164723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] PATH_INFO: `/v3/projects/ac368b03325e4eecade35db48402dc21` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.166018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac368b03325e4eecade35db48402dc21)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:29.167925 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:29.305876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac368b03325e4eecade35db48402dc21', 'request_id': 'req-f7247f89-55b6-49c3-a83d-31e4697a9326'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:29.311293 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac368b03325e4eecade35db48402dc21', 'request_id': 'req-f7247f89-55b6-49c3-a83d-31e4697a9326'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:29.317415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac368b03325e4eecade35db48402dc21', 'request_id': 'req-f7247f89-55b6-49c3-a83d-31e4697a9326'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:29.319985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:29.401253 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] The token cache is being invalidate because project ac368b03325e4eecade35db48402dc21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:29.401539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:29.402224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-f7247f89-55b6-49c3-a83d-31e4697a9326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:29.403857 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1345/2687] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:29 2022] DELETE /identity/v3/projects/ac368b03325e4eecade35db48402dc21 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:29.408653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.408944 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.409219 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.422842 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:29.428660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:29.429854 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:29.488234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1343/2688] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:29.496551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.497036 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.497429 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.516481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:29.517722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:29.527330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1346/2689] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:29.530498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:29.549884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361df945-4a8e-4136-8c15-43de6998b3ac None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:29.549884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:29.549884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.549884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.550312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.550312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:29.550312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:29.550312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:29.563269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1344/2690] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:29.574909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:29.584968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1daaf9-4c59-43eb-82ba-7dd131a8d9c0 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:29.590160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:29.591101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.591389 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.591664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.592359 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:29.594304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:29.712346 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1347/2691] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:29 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:29.716681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:29.727683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926fa2-1b08-429f-a070-3fa64ceef0ba admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:29.783029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:29.783695 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.783983 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.784257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.785022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:29.787630 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:29.956413 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1345/2692] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:29 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:29.961005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:29.971309 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef203ae8-22df-4fc4-8806-80ae5696fdd2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:29.976735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:29.977385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:29.977669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:29.977940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:29.978658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:29.980265 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:29.990398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1348/2693] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:29.994105 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.005152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f292c9-77d7-40ca-b74b-a51501aa4e02 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.010098 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.010765 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.011051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.011448 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.012463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.014103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327208-4712-40e8-b9ec-47e4523f1a71 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.025092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1346/2694] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:30.029358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.039344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f57563-9f6c-4f8b-b5c9-636557ab61ba admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.044143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.045040 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.045323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.045597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] PATH_INFO: `/v3/projects/30a32a8a51d34034a6b6d2826bb0b3cd/users/509048163d0344e386c87c9a18dd2365/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.059964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a32a8a51d34034a6b6d2826bb0b3cd, user_id=509048163d0344e386c87c9a18dd2365, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.062641 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.126914 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1349/2695] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:30 2022] PUT /identity/v3/projects/30a32a8a51d34034a6b6d2826bb0b3cd/users/509048163d0344e386c87c9a18dd2365/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:30.138543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.139030 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.140068 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.166586 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] MFA Rules not processed for user `509048163d0344e386c87c9a18dd2365`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:30.167813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:30.234411 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1347/2696] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:30.238855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.303127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fc61-e36c-4422-9302-31724d8991d4 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.307780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.308439 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.308752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.309028 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.309717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.311617 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.431234 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1350/2697] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:30 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:30.435238 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.440701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:30.446329 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9bbdb-ded3-48c9-9571-7833dbd2dae3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.501318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.501974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.502253 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.502526 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.503198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.505170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.632452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1348/2698] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:30 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:30.636475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.647483 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc0274-a403-483f-8ec2-967684569c72 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.652139 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.652843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.653141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.653514 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.654243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.655841 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.667074 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1351/2699] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:30.671013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.681222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863842c-66dd-4b85-b430-c493ee5f6d9d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.685899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.686496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.686794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.687056 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.687783 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.689399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596e18-b555-423c-8ba5-fcfbcc5b5775 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.699284 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1349/2700] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:30.703112 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.713405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861f23-53e5-44b2-97a2-9174de21dc0e admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.717851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.718706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.718995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.719264 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] PATH_INFO: `/v3/projects/4b680400e87746b89d616ebc8bf71be4/users/34a2d8675c40479bb5cd0761c923e7e3/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.734013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b680400e87746b89d616ebc8bf71be4, user_id=34a2d8675c40479bb5cd0761c923e7e3, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.736643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:30.812476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1352/2701] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:30 2022] PUT /identity/v3/projects/4b680400e87746b89d616ebc8bf71be4/users/34a2d8675c40479bb5cd0761c923e7e3/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:30.823145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.823623 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.823748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.850492 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] MFA Rules not processed for user `34a2d8675c40479bb5cd0761c923e7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:30.851692 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:30.916521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1350/2702] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:30.920991 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:30.964462 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:30.983851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a8c5-4603-468e-8fd4-287b0a592986 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:30.988502 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:30.989210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:30.989495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:30.989774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:30.990459 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:30.992353 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.084076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1353/2703] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:30 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:31.088163 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.098715 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8512d81-75ba-46ea-a041-2e8026e75257 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.154229 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.154880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.155165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.155435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.156122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.158055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.236853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:31.242365 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1351/2704] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:31 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:31.246317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.257533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d419c9a-b899-436f-b08b-09f51587e69a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.262163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.262863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.263148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.263426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.264151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.265779 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.276335 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1354/2705] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:31.280637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.290546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab310-6c05-4119-91ba-7aecb3b3fc3f admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.295737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.296622 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.296910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.297187 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] PATH_INFO: `/v3/projects/79d96020d15d4b0f8670f4026408dfab/users/ea245e79a4cf4d90a77db0a275df22af/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.312062 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d96020d15d4b0f8670f4026408dfab, user_id=ea245e79a4cf4d90a77db0a275df22af, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.314726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.384992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1352/2706] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:31 2022] PUT /identity/v3/projects/79d96020d15d4b0f8670f4026408dfab/users/ea245e79a4cf4d90a77db0a275df22af/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:31.389023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.399878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b0bb-5011-4735-b937-3cbe2687eb80 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.455816 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.456470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.456782 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.457055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.458125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.459708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ec97c-e394-431e-b40c-5a3cf252986e admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.469285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1355/2707] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:31.473997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.484455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e4288-c049-4ccb-aeae-2cac776ec231 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.489681 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.490494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.490775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.491049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] PATH_INFO: `/v3/projects/79d96020d15d4b0f8670f4026408dfab/users/ea245e79a4cf4d90a77db0a275df22af/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.493075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d96020d15d4b0f8670f4026408dfab, user_id=ea245e79a4cf4d90a77db0a275df22af, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.495674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.505747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1353/2708] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:31 2022] PUT /identity/v3/projects/79d96020d15d4b0f8670f4026408dfab/users/ea245e79a4cf4d90a77db0a275df22af/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:31.515574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.515861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.516138 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.541683 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] MFA Rules not processed for user `ea245e79a4cf4d90a77db0a275df22af`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:31.542871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:31.616087 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1356/2709] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:31.627293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.629055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:31.697132 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bbb6-4538-4284-bed1-a613739ca995 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.700809 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.701452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.701730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.702004 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.711180 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.712611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.722359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1354/2710] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:31.808722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:31.818322 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413181-7395-4206-8484-56e0b5f343ee None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:31.821959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:31.822597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:31.822875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:31.823146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:31.857843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:31.882021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:31.883466 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:31.892748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1357/2711] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:41.141987 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:41.143477 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:41.201647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9be2-272a-4080-83fe-4098063da527 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:41.205308 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:41.205982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:41.206270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:41.206545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:41.208288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:41.266313 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:41.267657 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:41.277193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1355/2712] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:41.966982 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=12 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:42.184477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:42.184477 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:42.265520 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb712ff8-f812-449d-98b3-b4c50b7d7990 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:42.272052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:42.272727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:42.273023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:42.273296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:42.275147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:42.289262 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:42.290906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:42.301884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1358/2713] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:42.874510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:42.883853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f437-0e67-46ca-89c1-d8d1e0e40af4 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:42.887650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:42.888288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:42.888645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:42.888945 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:42.890720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:42.900965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:42.902439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20ff91-0314-47b8-a964-1945ee6ce258 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:42.911369 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1356/2714] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:45.243392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:45.322415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76b9c-933a-4fec-af03-a24ffe3ded18 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:45.327428 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:45.328305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:45.328601 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:45.328895 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] PATH_INFO: `/v3/users/509048163d0344e386c87c9a18dd2365` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:45.330194 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=509048163d0344e386c87c9a18dd2365)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:45.332188 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:45.558141 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509048163d0344e386c87c9a18dd2365'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:45.565008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509048163d0344e386c87c9a18dd2365'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:45.565858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509048163d0344e386c87c9a18dd2365'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:45.630091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509048163d0344e386c87c9a18dd2365'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:45.630719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509048163d0344e386c87c9a18dd2365'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:45.655852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1359/2715] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:45 2022] DELETE /identity/v3/users/509048163d0344e386c87c9a18dd2365 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:45.659018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:45.659539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:45.659654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:45.672165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:45.679390 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:45.680697 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:45.743405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1357/2716] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:45.753103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:45.754645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:45.761338 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:45.814140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c4e5-ef0d-4752-a8bc-699c07bbcf45 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:45.817789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:45.818447 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:45.818741 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:45.819025 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:45.828485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:45.829963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:45.839354 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1360/2717] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:46.374663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:46.376409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:46.390425 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838f58f-766b-488a-913f-3caa2e2426c8 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:46.395350 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:46.397145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:46.397443 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:46.397718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] PATH_INFO: `/v3/projects/30a32a8a51d34034a6b6d2826bb0b3cd` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:46.399058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a32a8a51d34034a6b6d2826bb0b3cd)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:46.401178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:46.525726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a32a8a51d34034a6b6d2826bb0b3cd', 'request_id': 'req-b4a6125a-29f9-437a-a434-9838f5398ca7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:46.532267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a32a8a51d34034a6b6d2826bb0b3cd', 'request_id': 'req-b4a6125a-29f9-437a-a434-9838f5398ca7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:46.537864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a32a8a51d34034a6b6d2826bb0b3cd', 'request_id': 'req-b4a6125a-29f9-437a-a434-9838f5398ca7'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:46.687144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] The token cache is being invalidate because project 30a32a8a51d34034a6b6d2826bb0b3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:46.687832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:46.688387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:46.690043 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1358/2718] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:46 2022] DELETE /identity/v3/projects/30a32a8a51d34034a6b6d2826bb0b3cd => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:46.694589 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:46.705155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac24ee8-e75f-43ce-bde2-1037d15cb03d service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:46.771957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:46.772912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:46.773193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:46.773467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] PATH_INFO: `/v3/users/34a2d8675c40479bb5cd0761c923e7e3` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:46.774839 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a2d8675c40479bb5cd0761c923e7e3)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:46.776804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:46.928793 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:46.930239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:46.943167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6125a-29f9-437a-a434-9838f5398ca7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:46.947652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:46.948533 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:46.948844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:46.949118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] PATH_INFO: `/v3/users/207d444f601a4dec9c6321d88288bc80` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:46.950336 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=207d444f601a4dec9c6321d88288bc80)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:46.952240 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:47.032316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a2d8675c40479bb5cd0761c923e7e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.039787 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a2d8675c40479bb5cd0761c923e7e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.040451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a2d8675c40479bb5cd0761c923e7e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.104018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a2d8675c40479bb5cd0761c923e7e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.104778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c9c77753-d8a5-4c72-8f3d-1f61f53c9d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a2d8675c40479bb5cd0761c923e7e3'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.123118 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1361/2719] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:46 2022] DELETE /identity/v3/users/34a2d8675c40479bb5cd0761c923e7e3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:47.169440 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207d444f601a4dec9c6321d88288bc80'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.197093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207d444f601a4dec9c6321d88288bc80'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.321934 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207d444f601a4dec9c6321d88288bc80'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.322782 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207d444f601a4dec9c6321d88288bc80'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.323584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207d444f601a4dec9c6321d88288bc80'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.348537 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1359/2720] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:46 2022] DELETE /identity/v3/users/207d444f601a4dec9c6321d88288bc80 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:47.353205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:47.353525 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:47.353829 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:47.375664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:47.375664 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:47.404947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:47.452275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1362/2721] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:47.463734 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:47.465202 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:47.470559 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:47.527996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c0ad-48b4-4320-b907-b337898ae4f6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:47.532212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:47.532546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:47.532907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:47.533185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:47.543876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:47.545327 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:47.556088 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1360/2722] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:47.663167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:47.664604 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:47.676542 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6b8dc-aed0-415b-8ff8-88a278ce35e8 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:47.681804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:47.682677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:47.682972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:47.683254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] PATH_INFO: `/v3/projects/4b680400e87746b89d616ebc8bf71be4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:47.684474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b680400e87746b89d616ebc8bf71be4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:47.686618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:47.867004 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b680400e87746b89d616ebc8bf71be4', 'request_id': 'req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.872688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b680400e87746b89d616ebc8bf71be4', 'request_id': 'req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.879136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b680400e87746b89d616ebc8bf71be4', 'request_id': 'req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.972403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] The token cache is being invalidate because project 4b680400e87746b89d616ebc8bf71be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:47.972748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.973437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:47.975781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1363/2723] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:47 2022] DELETE /identity/v3/projects/4b680400e87746b89d616ebc8bf71be4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:47.979777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:47.989648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047fbe-d4d0-4f3f-93cc-c30caaf490f6 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.050281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.051148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.051428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.051702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] PATH_INFO: `/v3/users/ea245e79a4cf4d90a77db0a275df22af` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.053547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea245e79a4cf4d90a77db0a275df22af)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.055454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.089104 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.090551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.104844 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff67d-5b6e-4636-a8c4-60830a6ef373 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.110319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.111225 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.111524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.111774 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] PATH_INFO: `/v3/projects/cf690ad6c5694ad0a51944572935e117` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.113008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf690ad6c5694ad0a51944572935e117)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.115137 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.231971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea245e79a4cf4d90a77db0a275df22af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.238998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea245e79a4cf4d90a77db0a275df22af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.293368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf690ad6c5694ad0a51944572935e117', 'request_id': 'req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.298939 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf690ad6c5694ad0a51944572935e117', 'request_id': 'req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.306506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf690ad6c5694ad0a51944572935e117', 'request_id': 'req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.327318 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea245e79a4cf4d90a77db0a275df22af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.327942 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea245e79a4cf4d90a77db0a275df22af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.328552 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-e7d6f77c-1d91-4d75-bb68-cfe5b88cf21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea245e79a4cf4d90a77db0a275df22af'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.346600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1361/2724] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:47 2022] DELETE /identity/v3/users/ea245e79a4cf4d90a77db0a275df22af => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:48.431176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] The token cache is being invalidate because project cf690ad6c5694ad0a51944572935e117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:48.431586 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.432314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0e7b7fa4-cb0d-46ad-b980-ab77f8823452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:48.434295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1364/2725] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:48 2022] DELETE /identity/v3/projects/cf690ad6c5694ad0a51944572935e117 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:48.439090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.439914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.440950 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.460026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:48.461338 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.481606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:48.522244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1362/2726] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:48.530161 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.530457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.530760 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.548694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:48.549915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.558518 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1365/2727] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:48.563031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.578805 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9d55a-145f-4b2e-abe2-b8405daaf31d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.584794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.585757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.587185 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.587639 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.589025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:48.589452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.591473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.607454 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1363/2728] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:48.621066 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.630666 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fc82b-c460-4ab2-be39-5005ad629e2e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.636361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.637070 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.637371 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.637627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.638330 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.640275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.735263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.737126 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.741208 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1366/2729] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:48 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:48.745079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.751316 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:48.794167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84309bf9-8ab7-498c-a25f-b4d470686f3a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.807960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151f39-7e80-4bde-98a0-de98e853291c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.812831 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.813501 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.813784 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.814061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.814744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.816731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.845515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.846138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.846466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.846688 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.848505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.861853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.863304 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.873970 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1364/2730] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:39:48.874778 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.876076 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:48.888270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c671-2412-4d01-9b7a-dc678886eac1 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.893393 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.894354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.894746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.895207 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] PATH_INFO: `/v3/projects/79d96020d15d4b0f8670f4026408dfab` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.896562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d96020d15d4b0f8670f4026408dfab)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.898849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.911022 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1367/2731] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:48 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:48.915307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.926612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23191379-0e71-4433-8dc6-68d5f74e75d6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.932277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.933060 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.933437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.933781 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.934575 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.936252 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.946594 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1368/2732] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:48.950539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.960562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d16d3-f35a-4f51-a3c3-4172987d2945 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:48.965509 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:48.966201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:48.966488 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:48.966788 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:48.967578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:48.969268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:48.984082 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1369/2733] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:48.985092 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:48.995563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480065a7-5cc0-4cd9-8003-d595701d6e42 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.000299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.001193 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.001474 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.001752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] PATH_INFO: `/v3/projects/8c26e375cbb2421b8c8e34653f7d2f38/users/a446f2a301a04fa485094d7be5219a07/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.019307 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c26e375cbb2421b8c8e34653f7d2f38, user_id=a446f2a301a04fa485094d7be5219a07, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.022088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e8f86-0805-4ba8-b706-82ef771c0ba1 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.093636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1370/2734] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:48 2022] PUT /identity/v3/projects/8c26e375cbb2421b8c8e34653f7d2f38/users/a446f2a301a04fa485094d7be5219a07/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:49.098739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32f3a-4fa7-449f-bb15-3bb04398ba2e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.099609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32f3a-4fa7-449f-bb15-3bb04398ba2e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.103263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32f3a-4fa7-449f-bb15-3bb04398ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.104177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d96020d15d4b0f8670f4026408dfab', 'request_id': 'req-c447d89c-d05a-407e-851d-7973df583ba1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:49.111813 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d96020d15d4b0f8670f4026408dfab', 'request_id': 'req-c447d89c-d05a-407e-851d-7973df583ba1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:49.117786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d96020d15d4b0f8670f4026408dfab', 'request_id': 'req-c447d89c-d05a-407e-851d-7973df583ba1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:49.126730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-a9b32f3a-4fa7-449f-bb15-3bb04398ba2e None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:49.128013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a9b32f3a-4fa7-449f-bb15-3bb04398ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:49.189179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1371/2735] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:49.201311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] The token cache is being invalidate because project 79d96020d15d4b0f8670f4026408dfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:49.201755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:49.202427 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:49.205387 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1365/2736] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:48 2022] DELETE /identity/v3/projects/79d96020d15d4b0f8670f4026408dfab => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:49.210754 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f408be-cb6f-40fb-8123-48197892dcf4 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.211305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f408be-cb6f-40fb-8123-48197892dcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.211593 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f408be-cb6f-40fb-8123-48197892dcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.232050 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d2f408be-cb6f-40fb-8123-48197892dcf4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:49.234224 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d2f408be-cb6f-40fb-8123-48197892dcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:49.234343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.235643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:49.242358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1372/2737] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:49.251981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa265-4604-4c53-968b-a74b16909143 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.252391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa265-4604-4c53-968b-a74b16909143 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.252735 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa265-4604-4c53-968b-a74b16909143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.272706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-096fa265-4604-4c53-968b-a74b16909143 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:49.274075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-096fa265-4604-4c53-968b-a74b16909143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:49.288693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1373/2738] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:49.289726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fa265-4604-4c53-968b-a74b16909143 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.293277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:49.305063 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fa265-4604-4c53-968b-a74b16909143 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.310960 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447d89c-d05a-407e-851d-7973df583ba1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.311116 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.311905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.312200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.312830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.313411 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:39:49.313718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.315780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.316570 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.319260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.319260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.319260 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.321419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:49.334994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1374/2739] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:39:49.339288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.340682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.342791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.356198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1366/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:49.356977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8dc07-bece-4d29-aa99-fe00e45aa58f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.362824 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.363776 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.364155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.364625 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.366263 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.368457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.498011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1375/2741] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:49.499216 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.508849 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db99004-745e-4700-9df2-b9fc80534875 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.515926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:49.569674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.570410 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.570700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.570979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.571670 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.573742 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.714049 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1367/2742] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:49.715736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.727578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a647786-5c27-454a-b081-7e01564e7d82 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.732203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.732915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.733201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.733476 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.734214 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.735873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.751369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1376/2743] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:49.761058 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.773005 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e51d41-5909-4d64-8573-aea9597133c0 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.778354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.779092 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.780370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.780370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.780713 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.782406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2796a3-0a3e-484d-8c15-c9b171bd2039 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.794456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1368/2744] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:49.798882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:49.811186 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712cb9-19bf-4723-9a38-486fb378a30f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:49.816091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:49.817146 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.817434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.817709 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] PATH_INFO: `/v3/projects/6e6b7ab24019452eb91e74ee2b8b4bdf/users/9df535087d824e42b5051e5b2d0f9b2b/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.836855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6b7ab24019452eb91e74ee2b8b4bdf, user_id=9df535087d824e42b5051e5b2d0f9b2b, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:49.839956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:49.950556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1377/2745] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:49 2022] PUT /identity/v3/projects/6e6b7ab24019452eb91e74ee2b8b4bdf/users/9df535087d824e42b5051e5b2d0f9b2b/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:49.963022 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:49.963349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:49.963677 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:49.991893 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] MFA Rules not processed for user `9df535087d824e42b5051e5b2d0f9b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:49.993275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:50.065171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1369/2746] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:50.069752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.131210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d768e1-a0a0-4c6c-b2c1-94cad44a4e26 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.136045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.136739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.137473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.137473 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.138049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.141200 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:50.452260 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1378/2747] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:50 2022] POST /identity/v3/projects => generated 373 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:50.453414 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.460388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:50.465165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743631dd-fe2e-48eb-bbfb-c619d8e742e0 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.525317 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.526358 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.526643 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.526920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.527830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.530283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:50.641755 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1370/2748] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:50 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:39:50.645879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.656888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8121-c2d1-41a5-be1e-200ff67698eb admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.661605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.662972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.663212 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.663463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.664197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.665836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:50.678094 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1379/2749] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:50.682228 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.692865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcea0ca-8349-4d0d-aabc-1ac95cf5b11e admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.697619 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.698513 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.698800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.699083 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] PATH_INFO: `/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504/users/da6a9340aea7436aa2b08e2f3970f8ea/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.717091 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c41b75c18d4d7590e89ff3b93e2504, user_id=da6a9340aea7436aa2b08e2f3970f8ea, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.719888 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:50.844787 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1371/2750] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:50 2022] PUT /identity/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504/users/da6a9340aea7436aa2b08e2f3970f8ea/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:50.850241 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.860814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6dba0-efa9-4c39-ac36-65ac384efe00 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.937506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.938673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.938975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.939257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.940154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.942362 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f7b69-126e-4613-aeef-8a39ce100b4c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:50.960285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1380/2751] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:50.963087 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:50.975994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797609-158e-4a25-858b-6c7955ae8161 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:50.983596 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:50.985223 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:50.985743 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:50.986122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] PATH_INFO: `/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504/users/da6a9340aea7436aa2b08e2f3970f8ea/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:50.988507 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c41b75c18d4d7590e89ff3b93e2504, user_id=da6a9340aea7436aa2b08e2f3970f8ea, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:50.992053 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:51.015106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1372/2752] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:50 2022] PUT /identity/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504/users/da6a9340aea7436aa2b08e2f3970f8ea/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:51.020728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9441246-e3e8-4a26-9860-20a64509b675 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:51.021076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9441246-e3e8-4a26-9860-20a64509b675 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:51.021392 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9441246-e3e8-4a26-9860-20a64509b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:51.052923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f9441246-e3e8-4a26-9860-20a64509b675 None None] MFA Rules not processed for user `da6a9340aea7436aa2b08e2f3970f8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:51.054199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f9441246-e3e8-4a26-9860-20a64509b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:51.081729 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:51.134526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1381/2753] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:39:51.148086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:51.149857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:51.229288 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924057-b8d7-4b72-bda3-f7bb5a9d585b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:51.233142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:51.233939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:51.241866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:51.241866 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:51.306025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:51.307626 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:51.317736 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1373/2754] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:56.408842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:56.409396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:56.409396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:56.444021 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] MFA Rules not processed for user `a446f2a301a04fa485094d7be5219a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:56.446023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:56.542207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1382/2755] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:56.546429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:56.548188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:56.566652 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:56.631724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba25c8-b63c-4a89-9f42-5b4b1aacf46d service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:56.637234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:56.637722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:56.638008 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:56.638290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:56.638994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:56.641204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:56.730252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1374/2756] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:39:56 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:56.734237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:56.748304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2cb1-6254-4f21-a705-9a32c3b9ad3e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:56.834655 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:56.835903 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:56.836695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:56.836695 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:56.837386 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:56.839996 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:56.866615 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:56.877647 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff587c4-fb46-4196-b8c7-010d72957faf admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:56.883246 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:56.900660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:56.900660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:56.900660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] PATH_INFO: `/v3/users/9df535087d824e42b5051e5b2d0f9b2b` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:56.900660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df535087d824e42b5051e5b2d0f9b2b)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:56.900660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:56.958553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1383/2757] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:39:56 2022] POST /identity/v3/users => generated 296 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:39:56.959836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:56.973080 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e9406-82d3-46bc-a18e-5802c96e5ed8 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:56.980874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:56.981555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:56.981842 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:56.982120 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:56.983687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:56.985433 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:56.998364 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1384/2758] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:39:57.004151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:57.018789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ac5ac-d621-455d-9ca1-4a8815207bb6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:57.023718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:57.024631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:57.024918 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:57.025198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] PATH_INFO: `/v3/projects/9ab7584c6e1b49248b951072bef46503/users/b669c71e4d1143b9a9c3259dfd5850be/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:57.042023 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab7584c6e1b49248b951072bef46503, user_id=b669c71e4d1143b9a9c3259dfd5850be, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:57.044719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:57.089923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df535087d824e42b5051e5b2d0f9b2b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:57.096856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df535087d824e42b5051e5b2d0f9b2b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:57.138535 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1385/2759] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:57 2022] PUT /identity/v3/projects/9ab7584c6e1b49248b951072bef46503/users/b669c71e4d1143b9a9c3259dfd5850be/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:57.142487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:57.147682 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:57.169406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df535087d824e42b5051e5b2d0f9b2b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:57.169406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df535087d824e42b5051e5b2d0f9b2b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:57.173072 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-feb42e78-712a-480b-8b5a-4201642575e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df535087d824e42b5051e5b2d0f9b2b'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:57.206747 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1375/2760] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:56 2022] DELETE /identity/v3/users/9df535087d824e42b5051e5b2d0f9b2b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:57.209663 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:57.210354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:57.210896 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:57.239159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:39:57.242751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:57.250192 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea318-c6bc-440e-b1f7-6b6a9fbf5a81 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:57.358071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:57.362947 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:57.363251 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:57.363543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:57.364310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:57.367935 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:57.399759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1386/2761] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:39:57.400738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:57.410533 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:57.435349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5f6a6-6744-4fb0-86e6-b4f07ca83aab admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:57.436389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1376/2762] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:39:57.441874 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:57.442882 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:57.443268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:57.443621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] PATH_INFO: `/v3/projects/9ab7584c6e1b49248b951072bef46503/users/b669c71e4d1143b9a9c3259dfd5850be/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:57.447348 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:57.448672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:57.449219 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab7584c6e1b49248b951072bef46503, user_id=b669c71e4d1143b9a9c3259dfd5850be, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:57.452545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:57.471604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:57.484482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1387/2763] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:39:57 2022] PUT /identity/v3/projects/9ab7584c6e1b49248b951072bef46503/users/b669c71e4d1143b9a9c3259dfd5850be/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:57.545510 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9982c-8e4a-4ec8-ad4b-80c2016daad7 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:57.617571 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:57.618373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:57.618574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:57.618825 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:57.719239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:57.721373 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:57.740214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1377/2764] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:39:58.383764 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:58.385475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:58.403539 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6c75e-3a2e-4427-80cb-ed537916818a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:58.408667 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:58.409694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:58.410033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:58.410394 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] PATH_INFO: `/v3/users/9db3cd25f4c94640854677709d10ccbe` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:58.411881 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db3cd25f4c94640854677709d10ccbe)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:58.417658 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:58.486057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:58.487870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:58.513543 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b5f57-3722-4bde-b9b8-18a24e42ee7b service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:58.526037 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:58.527654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:58.527952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:58.528995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] PATH_INFO: `/v3/projects/6e6b7ab24019452eb91e74ee2b8b4bdf` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:58.532917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6b7ab24019452eb91e74ee2b8b4bdf)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:58.534982 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:58.618410 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db3cd25f4c94640854677709d10ccbe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.634762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db3cd25f4c94640854677709d10ccbe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.642794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db3cd25f4c94640854677709d10ccbe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.700352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6b7ab24019452eb91e74ee2b8b4bdf', 'request_id': 'req-0531aad4-19a8-44c2-8bcf-e094045ae759'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.708328 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6b7ab24019452eb91e74ee2b8b4bdf', 'request_id': 'req-0531aad4-19a8-44c2-8bcf-e094045ae759'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.716762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6b7ab24019452eb91e74ee2b8b4bdf', 'request_id': 'req-0531aad4-19a8-44c2-8bcf-e094045ae759'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.738486 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db3cd25f4c94640854677709d10ccbe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.739497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db3cd25f4c94640854677709d10ccbe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.763547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1388/2765] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:58 2022] DELETE /identity/v3/users/9db3cd25f4c94640854677709d10ccbe => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:58.853780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:58.859779 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] The token cache is being invalidate because project 6e6b7ab24019452eb91e74ee2b8b4bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:58.861391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.861391 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:58.866587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1378/2766] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:58 2022] DELETE /identity/v3/projects/6e6b7ab24019452eb91e74ee2b8b4bdf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:58.868761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:58.871013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:58.883154 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:58.953964 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfec3e7-2044-4576-b0a7-b16d4781b87c admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:58.959543 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:58.960360 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:58.960674 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:58.960972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] PATH_INFO: `/v3/users/da6a9340aea7436aa2b08e2f3970f8ea` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:58.962178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6a9340aea7436aa2b08e2f3970f8ea)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:58.964119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:59.203433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:59.206311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:59.281376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531aad4-19a8-44c2-8bcf-e094045ae759 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:59.290298 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:59.291200 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:59.291483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:59.291874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] PATH_INFO: `/v3/projects/ae0e917f30d3430bb088ca3d0afe1a02` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:59.293044 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0e917f30d3430bb088ca3d0afe1a02)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:59.294989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:59.363492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6a9340aea7436aa2b08e2f3970f8ea'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.373257 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6a9340aea7436aa2b08e2f3970f8ea'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.374320 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6a9340aea7436aa2b08e2f3970f8ea'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.428184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0e917f30d3430bb088ca3d0afe1a02', 'request_id': 'req-ff77f6df-00f5-4180-b40d-392356adbf54'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.437997 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0e917f30d3430bb088ca3d0afe1a02', 'request_id': 'req-ff77f6df-00f5-4180-b40d-392356adbf54'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.447923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0e917f30d3430bb088ca3d0afe1a02', 'request_id': 'req-ff77f6df-00f5-4180-b40d-392356adbf54'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.458557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6a9340aea7436aa2b08e2f3970f8ea'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.460031 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6a9340aea7436aa2b08e2f3970f8ea'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.487698 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1389/2767] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:58 2022] DELETE /identity/v3/users/da6a9340aea7436aa2b08e2f3970f8ea => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:39:59.552110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] The token cache is being invalidate because project ae0e917f30d3430bb088ca3d0afe1a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:39:59.553299 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.554455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.556881 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1379/2768] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:39:59 2022] DELETE /identity/v3/projects/ae0e917f30d3430bb088ca3d0afe1a02 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:39:59.560668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:39:59.562304 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:39:59.646598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6734a3-458d-446c-8998-933eb8d8a7b7 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:39:59.651983 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:39:59.652905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:39:59.653185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:39:59.653458 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] PATH_INFO: `/v3/users/cf564ae80a0a47ba93acb18832cd250d` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:39:59.655217 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf564ae80a0a47ba93acb18832cd250d)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:39:59.658538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:39:59.893153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:39:59.902014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf564ae80a0a47ba93acb18832cd250d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.923125 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf564ae80a0a47ba93acb18832cd250d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:39:59.926451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf564ae80a0a47ba93acb18832cd250d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.001399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf564ae80a0a47ba93acb18832cd250d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.005797 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf564ae80a0a47ba93acb18832cd250d'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.054079 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1390/2769] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:39:59 2022] DELETE /identity/v3/users/cf564ae80a0a47ba93acb18832cd250d => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:00.083392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:00.085614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.175074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f6df-00f5-4180-b40d-392356adbf54 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:00.187737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:00.189397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.234252 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:00.297326 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:00.301417 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:00.301876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:00.303808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] PATH_INFO: `/v3/users/aeb638796248423880622b4cc598976e` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:00.309055 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb638796248423880622b4cc598976e)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:00.311250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:00.374207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38d621-5c4c-4322-9f25-1120ff08c826 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:00.379068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:00.379950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:00.380235 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:00.380744 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] PATH_INFO: `/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:00.382000 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c41b75c18d4d7590e89ff3b93e2504)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:00.383956 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:00.617521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c41b75c18d4d7590e89ff3b93e2504', 'request_id': 'req-110d0969-efbb-4706-b3ad-52af930ad383'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.621761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb638796248423880622b4cc598976e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.626960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c41b75c18d4d7590e89ff3b93e2504', 'request_id': 'req-110d0969-efbb-4706-b3ad-52af930ad383'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.632874 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb638796248423880622b4cc598976e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.638435 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c41b75c18d4d7590e89ff3b93e2504', 'request_id': 'req-110d0969-efbb-4706-b3ad-52af930ad383'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.711179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb638796248423880622b4cc598976e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.712167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb638796248423880622b4cc598976e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.712912 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-acc221ac-c3c6-4984-b05b-07a2af58c34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb638796248423880622b4cc598976e'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.747020 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1380/2770] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:00 2022] DELETE /identity/v3/users/aeb638796248423880622b4cc598976e => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:00.748241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:00.748678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:00.749218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:00.771999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:00.775709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.775851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] The token cache is being invalidate because project a8c41b75c18d4d7590e89ff3b93e2504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:00.775851 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.776390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-110d0969-efbb-4706-b3ad-52af930ad383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:00.778738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1391/2771] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:00 2022] DELETE /identity/v3/projects/a8c41b75c18d4d7590e89ff3b93e2504 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:00.783745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:00.789769 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:00.790484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:00.828231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:00.829799 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.931604 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1381/2772] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:40:00.932685 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:00.937972 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.945131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1392/2773] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:00.946178 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:00.949054 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:00.954512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434473b2-2431-4e1d-b9d4-92ad4b8ddffe None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:00.961016 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:00.962403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:00.962403 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:00.965401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] PATH_INFO: `/v3/projects/66b4551152af4d9fad50672dec4ad3a1` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:00.968352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b4551152af4d9fad50672dec4ad3a1)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:00.972645 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:01.052434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab14a-bbd0-45c4-b587-21128ecbf2e3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:01.058651 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:01.059277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.059563 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.059848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.072107 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:01.077485 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bd16e-85d0-4bb3-bd31-73e300a836d3 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:01.093745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1393/2774] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:01.095633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.095946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.096337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.116232 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b4551152af4d9fad50672dec4ad3a1', 'request_id': 'req-1012a437-edb8-4f69-bcf6-a863e47d45c2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:01.117761 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:01.119231 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:01.126564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b4551152af4d9fad50672dec4ad3a1', 'request_id': 'req-1012a437-edb8-4f69-bcf6-a863e47d45c2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:01.148054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b4551152af4d9fad50672dec4ad3a1', 'request_id': 'req-1012a437-edb8-4f69-bcf6-a863e47d45c2'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:01.245939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] The token cache is being invalidate because project 66b4551152af4d9fad50672dec4ad3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:01.246341 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:01.248856 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:01.260703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1382/2775] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:00 2022] DELETE /identity/v3/projects/66b4551152af4d9fad50672dec4ad3a1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:01.272503 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1394/2776] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:01.272899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:01.285880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a437-edb8-4f69-bcf6-a863e47d45c2 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:01.296995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:01.304611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:01.310056 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:01.325791 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1383/2777] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:40:01.330621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:01.359915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41cb84-60c2-45eb-85e6-535e21eb7866 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:01.365906 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:01.366605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.366889 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.367169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.370418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:01.372740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:01.796213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1395/2778] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:01 2022] POST /identity/v3/projects => generated 381 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:01.799730 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:01.815444 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4aaf6-3986-4593-a19e-c2f399f28090 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:01.835985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:01.846272 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:01.909245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:01.915609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.916051 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.916339 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.917177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:01.919294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:01.950169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:01.954923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289b29-62b7-49d2-a374-9cfc61af70d6 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:01.962753 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:01.963642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:01.963924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:01.964834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] PATH_INFO: `/v3/projects/480ccd8e209d40a5963c98c753e6af53` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:01.965905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=480ccd8e209d40a5963c98c753e6af53)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:01.968385 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.024917 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1384/2779] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:01 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:02.029498 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.049095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cfaeb-51b0-4914-be59-2103bc347e22 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.055650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.056453 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.056965 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.057249 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.057988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.059706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.074996 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1385/2780] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:02.080306 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.096340 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7b73-8c9a-4754-8372-e53301dc29bd admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.097029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480ccd8e209d40a5963c98c753e6af53', 'request_id': 'req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:02.108677 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480ccd8e209d40a5963c98c753e6af53', 'request_id': 'req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:02.125803 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480ccd8e209d40a5963c98c753e6af53', 'request_id': 'req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:02.176281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.177353 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.177684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.178029 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.178833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.180696 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.197034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1386/2781] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:02.201326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] The token cache is being invalidate because project 480ccd8e209d40a5963c98c753e6af53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:02.201479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.201591 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:02.202151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-ec96640f-b03b-4b06-8dbe-aec0724d2cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:02.207966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1396/2782] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:01 2022] DELETE /identity/v3/projects/480ccd8e209d40a5963c98c753e6af53 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:02.209644 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0933a3-af3c-42e7-aab3-4227c9b65787 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.210051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0933a3-af3c-42e7-aab3-4227c9b65787 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.210430 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0933a3-af3c-42e7-aab3-4227c9b65787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.215093 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655556a1-86f2-4aa4-9974-c616c230d582 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.223354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.225146 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.225429 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.225706 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] PATH_INFO: `/v3/projects/a0a185c667b14f19aee6427e4f91af8d/users/2e99a37e30d547b992b4ede147f684a4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.231526 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-1b0933a3-af3c-42e7-aab3-4227c9b65787 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:02.233288 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1b0933a3-af3c-42e7-aab3-4227c9b65787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:02.242725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1397/2783] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:02.250368 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a185c667b14f19aee6427e4f91af8d, user_id=2e99a37e30d547b992b4ede147f684a4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.252482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.257128 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.257663 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.261475 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4fd1b-2655-47a3-b864-03250368d300 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.280071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:02.281684 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:02.291325 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1398/2784] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:02.295390 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.304885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a3e1-0030-43d0-a281-b5bbc890dc2c None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.311242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.311242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.311242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.311242 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.312504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:40:02.312668 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.314343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.327190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1399/2785] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:40:02.335182 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.345597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e190-1e35-41ee-abbc-c825f8c3afe2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.350826 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.351480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.351757 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.352034 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.352765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.361536 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.365989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1387/2786] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:02 2022] PUT /identity/v3/projects/a0a185c667b14f19aee6427e4f91af8d/users/2e99a37e30d547b992b4ede147f684a4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:02.370077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.370439 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.370784 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.393103 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:02.394456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:02.467711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1388/2787] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:02.480455 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1400/2788] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:40:02.480808 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.481406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:02.482633 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.571736 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5a2bd-1a46-4bf8-9dfd-97efcfd3705a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.577791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.578738 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.579270 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.579648 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.580463 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.582869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.587988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ccc23-ebf6-43cb-8b28-f51b002f3d27 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.644234 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.645316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.645316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.652998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.659257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.660819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.671948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1389/2789] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:40:02.681934 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1401/2790] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:02.686071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.704838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6436d-5eba-4dc5-8610-a46097ccce4d service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.710614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.711283 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.711572 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.711855 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.712678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.719474 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.732935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1390/2791] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:02.736894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.748451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f765d2-58f3-4582-aef4-9498f38331a2 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.754484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.755163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.755475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.755808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.756598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.758423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a6602-d56b-44f1-93f7-b049f2b0cb25 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.769064 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1402/2792] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:02.773768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:02.784487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb3eeb-3710-48ee-92bb-b5cd3a399365 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:02.790278 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:02.791311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.791625 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.791904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] PATH_INFO: `/v3/projects/437187dfa5b3481690a7723a6a351fa2/users/2009b7b909c14798a852d99f8b9ba85f/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.797017 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:02.809851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437187dfa5b3481690a7723a6a351fa2, user_id=2009b7b909c14798a852d99f8b9ba85f, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:02.813199 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:02.867349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1391/2793] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:02 2022] PUT /identity/v3/projects/437187dfa5b3481690a7723a6a351fa2/users/2009b7b909c14798a852d99f8b9ba85f/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:02.878391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:02.878688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:02.878975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:02.911399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] MFA Rules not processed for user `2009b7b909c14798a852d99f8b9ba85f`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:02.913204 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:02.990277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1403/2794] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:02.995801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.065136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8e8c-039a-46f7-a219-629ff1ef0969 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.071953 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.072648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.072935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.073218 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.073956 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.075907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:03.201962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:03.245941 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1392/2795] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:02 2022] POST /identity/v3/projects => generated 375 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:40:03.249960 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.262796 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d1636-3f31-453c-8cb1-1fa1f27aaacf None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.328365 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.329415 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.330010 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.330450 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.331334 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.334913 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:03.413589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.425715 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1404/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:03 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:03.430876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0f9a-0ffb-40d6-910d-9be61c25a3be admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.437989 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.439310 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.439761 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.441700 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.442648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.445177 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:03.462217 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1393/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:03.463836 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.475706 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa397-efa5-47b6-8b23-5d7056c3eb4d admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.481059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.483722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.483722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.483722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.483722 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.484986 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df28d8-d691-40fb-a2dc-e4e7b53113de admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:03.499277 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1405/2798] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:03.502548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.505298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:03.513919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288e36-d913-4d6a-b3fe-e27eee2af61b admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.519312 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.520281 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.521481 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.521641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] PATH_INFO: `/v3/projects/451c053085964c438a9f26012163c276/users/eb2aec4b7f04450fa64c9b9f266107ea/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.539003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=451c053085964c438a9f26012163c276, user_id=eb2aec4b7f04450fa64c9b9f266107ea, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.541833 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:03.648224 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1394/2799] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:03 2022] PUT /identity/v3/projects/451c053085964c438a9f26012163c276/users/eb2aec4b7f04450fa64c9b9f266107ea/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:03.659148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.659545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.659917 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.706972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] MFA Rules not processed for user `eb2aec4b7f04450fa64c9b9f266107ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:03.708024 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:03.801405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1406/2800] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:03.806442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:03.890166 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f053e-366e-4464-8082-9b1b7f3e7c90 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:03.896479 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:03.897751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:03.898227 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:03.898614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:03.899564 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:03.901961 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.019644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1395/2801] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:03 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:04.023723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.038238 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd52d60-95d4-461c-a059-3ebdc94d53d3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.111163 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.111886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.112232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.112572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.113347 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.115672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.219810 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1407/2802] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:04 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:04.224193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.235395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674794-42ab-4472-90aa-8deb88652d52 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.241334 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.242003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.242286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.242560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.243316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.244947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.259469 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1396/2803] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:04.260693 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.273088 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303705-317e-4bc6-a68b-707fd8b88d93 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.278932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.278932 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.279285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.279480 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] PATH_INFO: `/v3/projects/1e33a075f2844d918dd01119e81732fe/users/b4a29059171b444dafc391f8bb5afa11/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.297730 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e33a075f2844d918dd01119e81732fe, user_id=b4a29059171b444dafc391f8bb5afa11, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.300845 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.315821 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:04.401445 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1408/2804] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:04 2022] PUT /identity/v3/projects/1e33a075f2844d918dd01119e81732fe/users/b4a29059171b444dafc391f8bb5afa11/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:04.405337 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.416909 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672391ae-f16e-4649-8f34-0080fe38745a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.480027 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.480768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.481033 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.481307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.482041 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.483660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8730e7-4be4-49e8-9162-aa5b11d85a73 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.494780 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1397/2805] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:04.498905 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.509672 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d234-3f17-4a80-82f2-458b04ccf382 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.514471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.515303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.515585 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.515859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] PATH_INFO: `/v3/projects/1e33a075f2844d918dd01119e81732fe/users/b4a29059171b444dafc391f8bb5afa11/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.517863 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e33a075f2844d918dd01119e81732fe, user_id=b4a29059171b444dafc391f8bb5afa11, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.520519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.532095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1409/2806] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:04 2022] PUT /identity/v3/projects/1e33a075f2844d918dd01119e81732fe/users/b4a29059171b444dafc391f8bb5afa11/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:40:04.602193 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63453-face-4554-9f8b-0cc7a02e9893 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.602487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63453-face-4554-9f8b-0cc7a02e9893 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.602766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63453-face-4554-9f8b-0cc7a02e9893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.635038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-2ef63453-face-4554-9f8b-0cc7a02e9893 None None] MFA Rules not processed for user `b4a29059171b444dafc391f8bb5afa11`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:04.637555 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-2ef63453-face-4554-9f8b-0cc7a02e9893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:04.760239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1398/2807] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:04.781804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.781804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:04.833188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.833188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.833188 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.850108 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860466-696a-4596-9f5c-af8fca828341 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.864500 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:04.865232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:04.865357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] MFA Rules not processed for user `2e99a37e30d547b992b4ede147f684a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:04.865511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:04.865783 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:04.866931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:04.956240 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:04.959278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fdd03-a13e-420f-851b-41f2aa606157 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:04.962831 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1399/2808] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:40:04.973724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:04.975395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1410/2809] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:04.977797 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:04.996034 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc9c98-66e2-4654-a366-dde8f5afc48c None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:04.999885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:05.000544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:05.000894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:05.001174 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:05.023392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:05.023392 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:05.033293 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1400/2810] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:40:05.253343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:05.255702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0fff-6985-4b9d-82fd-9e03e73482a6 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:05.256339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0fff-6985-4b9d-82fd-9e03e73482a6 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:05.256665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0fff-6985-4b9d-82fd-9e03e73482a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:05.292994 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-b24c0fff-6985-4b9d-82fd-9e03e73482a6 None None] MFA Rules not processed for user `b669c71e4d1143b9a9c3259dfd5850be`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:05.294237 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-b24c0fff-6985-4b9d-82fd-9e03e73482a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:05.350672 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:05.404612 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1411/2811] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:05.420316 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:05.421826 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:05.521221 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416baaca-a931-460b-87ac-06c0fe1c5654 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:05.526819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:05.527466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:05.527745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:05.528018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:05.529843 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:05.592738 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:05.597268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:05.604222 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1401/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:05.945537 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:06.246889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:08.019041 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:08.019562 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:08.019669 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:08.038492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] MFA Rules not processed for user `2e99a37e30d547b992b4ede147f684a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:08.041254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:08.048846 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1412/2813] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:08.198637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:08.212634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1fef-1c17-4b45-bcea-8def67a9af4c service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:08.217580 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:08.218354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:08.218740 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:08.219075 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:08.311398 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:08.329071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:08.331364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:08.357955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1402/2814] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:08.416460 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3db700-e8b8-40ee-9962-3844d9046a9e None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:08.421957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:08.423434 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:08.424164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:08.425375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:08.429702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:08.441013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:08.442375 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:08.452479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1413/2815] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:09.965590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:09.967198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:10.200388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813291-e816-4c92-9b39-9a3c3207c342 service cinder] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:10.200388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:10.200388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:10.200388 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:10.200812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:10.200812 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:10.212638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:10.214239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:10.226653 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1403/2816] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:10.281356 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:10.291795 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaef2b-9a6d-4410-99b4-46c8d1c78b77 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:10.295102 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:10.295733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:10.296018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:10.296303 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:10.307399 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:10.309203 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfed9e-3fc2-4655-beae-4db20c802347 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:10.321213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1414/2817] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:14.917758 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:14.947712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=100 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:15.007100 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f0bd0-b877-4e73-8076-639bf4d9806d service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:15.012837 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:15.017487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:15.017487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:15.017487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] PATH_INFO: `/v3/users/2009b7b909c14798a852d99f8b9ba85f` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:15.018722 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2009b7b909c14798a852d99f8b9ba85f)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:15.020683 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:15.158291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2009b7b909c14798a852d99f8b9ba85f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:15.165459 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2009b7b909c14798a852d99f8b9ba85f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:15.206198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2009b7b909c14798a852d99f8b9ba85f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:15.206737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2009b7b909c14798a852d99f8b9ba85f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:15.207330 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2009b7b909c14798a852d99f8b9ba85f'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:15.225107 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1404/2818] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:14 2022] DELETE /identity/v3/users/2009b7b909c14798a852d99f8b9ba85f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:15.229794 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:15.230339 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:15.230506 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:15.273059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:15.274289 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:15.335296 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1415/2819] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:15.348988 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:15.350431 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:15.443857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed783-61b3-4ffb-a89c-fbfe054fddd1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:15.448042 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:15.448733 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:15.449013 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:15.449292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:15.459110 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:15.460614 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:15.464772 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:15.470973 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1405/2820] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:40:16.070859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:16.072373 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:16.091739 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=100 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:16.110355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3f881-77fd-40b7-a97c-da340c550ae0 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:16.115283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:16.116136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:16.116418 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:16.116726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] PATH_INFO: `/v3/projects/437187dfa5b3481690a7723a6a351fa2` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:16.117909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=437187dfa5b3481690a7723a6a351fa2)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:16.119828 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:16.246766 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437187dfa5b3481690a7723a6a351fa2', 'request_id': 'req-7739797d-f2f0-4e16-8822-1b59dfa3e73a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.252279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437187dfa5b3481690a7723a6a351fa2', 'request_id': 'req-7739797d-f2f0-4e16-8822-1b59dfa3e73a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.258538 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437187dfa5b3481690a7723a6a351fa2', 'request_id': 'req-7739797d-f2f0-4e16-8822-1b59dfa3e73a'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.326213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] The token cache is being invalidate because project 437187dfa5b3481690a7723a6a351fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:16.326505 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.327162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.328768 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1416/2821] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:16 2022] DELETE /identity/v3/projects/437187dfa5b3481690a7723a6a351fa2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:16.332844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:16.354478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632924-8b74-4ca6-aee6-0ed84d93ea2c service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:16.419074 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:16.419955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:16.420239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:16.420515 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] PATH_INFO: `/v3/users/eb2aec4b7f04450fa64c9b9f266107ea` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:16.421737 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2aec4b7f04450fa64c9b9f266107ea)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:16.423668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:16.604590 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2aec4b7f04450fa64c9b9f266107ea'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.612315 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2aec4b7f04450fa64c9b9f266107ea'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.669566 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2aec4b7f04450fa64c9b9f266107ea'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.670355 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2aec4b7f04450fa64c9b9f266107ea'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.671018 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2aec4b7f04450fa64c9b9f266107ea'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:16.689455 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1406/2822] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:16 2022] DELETE /identity/v3/users/eb2aec4b7f04450fa64c9b9f266107ea => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:17.181426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:17.185621 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:17.202596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739797d-f2f0-4e16-8822-1b59dfa3e73a admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:17.275957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:17.276886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:17.277167 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:17.277438 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] PATH_INFO: `/v3/projects/451c053085964c438a9f26012163c276` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:17.278750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] RBAC: Authorizing `identity:delete_project(project_id=451c053085964c438a9f26012163c276)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:17.281019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:17.406926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '451c053085964c438a9f26012163c276', 'request_id': 'req-4d650fd4-4d3b-4439-80c6-e7772f9630be'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.415278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '451c053085964c438a9f26012163c276', 'request_id': 'req-4d650fd4-4d3b-4439-80c6-e7772f9630be'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.423278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '451c053085964c438a9f26012163c276', 'request_id': 'req-4d650fd4-4d3b-4439-80c6-e7772f9630be'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.504467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] The token cache is being invalidate because project 451c053085964c438a9f26012163c276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:17.504834 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.505565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.507565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1417/2823] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:17 2022] DELETE /identity/v3/projects/451c053085964c438a9f26012163c276 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:17.514595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:17.518367 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:17.532405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77574143-5133-44b4-a4f0-ae8651052ec6 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:17.589962 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:17.590829 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:17.591109 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:17.591385 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] PATH_INFO: `/v3/users/b4a29059171b444dafc391f8bb5afa11` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:17.592569 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a29059171b444dafc391f8bb5afa11)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:17.596345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:17.605952 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:17.821404 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a29059171b444dafc391f8bb5afa11'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.828360 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a29059171b444dafc391f8bb5afa11'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.892374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a29059171b444dafc391f8bb5afa11'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.893026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a29059171b444dafc391f8bb5afa11'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.893638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-1eb7702e-dc54-460c-844e-72d70d469b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a29059171b444dafc391f8bb5afa11'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:17.943279 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1407/2824] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:17 2022] DELETE /identity/v3/users/b4a29059171b444dafc391f8bb5afa11 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:18.379226 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:18.382055 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:18.400995 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650fd4-4d3b-4439-80c6-e7772f9630be admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:18.412746 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:18.474574 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:18.475467 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:18.475751 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:18.476030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] PATH_INFO: `/v3/projects/1e33a075f2844d918dd01119e81732fe` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:18.477243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e33a075f2844d918dd01119e81732fe)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:18.479351 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:18.618701 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e33a075f2844d918dd01119e81732fe', 'request_id': 'req-fe1c9366-927a-4f9b-bbfd-853277091290'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:18.624202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e33a075f2844d918dd01119e81732fe', 'request_id': 'req-fe1c9366-927a-4f9b-bbfd-853277091290'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:18.630547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e33a075f2844d918dd01119e81732fe', 'request_id': 'req-fe1c9366-927a-4f9b-bbfd-853277091290'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:18.727583 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] The token cache is being invalidate because project 1e33a075f2844d918dd01119e81732fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:18.728750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:18.729053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-fe1c9366-927a-4f9b-bbfd-853277091290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:18.730909 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1418/2825] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:18 2022] DELETE /identity/v3/projects/1e33a075f2844d918dd01119e81732fe => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:18.736460 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:18.736979 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:18.737081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:18.784038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:18.785271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:18.845401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1408/2826] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:18.854049 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:18.854343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:18.854619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:18.875002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:18.876605 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:18.886121 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1419/2827] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:18.890405 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:18.900080 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f566c01-3f40-4fa6-a37b-f4a6c1eac459 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:18.904600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:18.905257 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:18.905531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:18.905800 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:18.906518 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:40:18.906931 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:18.908509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:18.920544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1409/2828] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:40:18.925886 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:18.936091 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae4dfa-788a-478e-96cb-994a4f202945 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:18.940997 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:18.941645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:18.941926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:18.942199 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:18.942888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:18.944833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.031003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1420/2829] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:18 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:19.040089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.049968 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916976f-61e8-4b32-889f-acb5bb39b086 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.097620 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:19.107094 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.107746 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.108165 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.108442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.109171 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.111081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.172516 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1410/2830] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:19 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:19.176517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.188302 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d4fc8-1739-4805-b139-d77451be45ea admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.193151 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.194504 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.194786 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.196631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.196631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.197363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.207901 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1421/2831] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:19.211764 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.224104 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653a8c5-c73d-400e-a9db-0d4ea4a00fde admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.229788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.230638 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.230924 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.231345 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.232172 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.233841 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf1d13-8f2c-43f1-9146-1b6e6b1b15db admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.245985 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1411/2832] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:19.249825 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.260393 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e837e4-d608-497a-9f84-7e58102037a0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.264993 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.265858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.266140 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.266414 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] PATH_INFO: `/v3/projects/64abc33a14e24861b5edff29b8aeeb7d/users/4bdc37d42c1c476cbe495924cbe1dfa4/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.280423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=64abc33a14e24861b5edff29b8aeeb7d, user_id=4bdc37d42c1c476cbe495924cbe1dfa4, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.283095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.365110 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1422/2833] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:19 2022] PUT /identity/v3/projects/64abc33a14e24861b5edff29b8aeeb7d/users/4bdc37d42c1c476cbe495924cbe1dfa4/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:19.376025 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.376311 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.376600 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.402589 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] MFA Rules not processed for user `4bdc37d42c1c476cbe495924cbe1dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:19.403762 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:19.467071 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1412/2834] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:19.479016 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.481124 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:19.521702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:19.543962 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c0fe-92af-4a4b-b708-12a3889a51ba admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.547619 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.548266 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.548553 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.548864 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.558395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.559837 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f219ca-9543-48c2-b6b6-6f64a4573d7c service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:19.569005 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1423/2835] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:19.594290 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:19.661650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27be05-4af5-4c16-bd87-e4e14ce7da93 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:19.666684 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:19.667558 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:19.667839 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:19.668112 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] PATH_INFO: `/v3/users/4bdc37d42c1c476cbe495924cbe1dfa4` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:19.669420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdc37d42c1c476cbe495924cbe1dfa4)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:19.671357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:20.151241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdc37d42c1c476cbe495924cbe1dfa4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:20.158178 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdc37d42c1c476cbe495924cbe1dfa4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:20.234844 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdc37d42c1c476cbe495924cbe1dfa4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:20.235487 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdc37d42c1c476cbe495924cbe1dfa4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:20.236122 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdc37d42c1c476cbe495924cbe1dfa4'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:20.237483 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:20.254845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1413/2836] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:19 2022] DELETE /identity/v3/users/4bdc37d42c1c476cbe495924cbe1dfa4 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:20.259299 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:20.259614 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:20.259885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:20.276819 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:20.278027 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:20.336452 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1424/2837] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:20.346206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:20.347720 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:20.416277 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad8250-60f4-4c96-bbf5-f743176a55a7 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:20.419894 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:20.420525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:20.420838 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:20.421114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:20.430801 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:20.432220 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0bf16-1b6b-469b-aeee-e0ede4c32621 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:20.441995 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1414/2838] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:40:20.922003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:20.923471 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:20.936949 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ee45d-1bf5-4942-81d2-31cd1dbdad61 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:20.942009 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:20.942871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:20.943160 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:20.943977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] PATH_INFO: `/v3/projects/64abc33a14e24861b5edff29b8aeeb7d` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:20.945210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=64abc33a14e24861b5edff29b8aeeb7d)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:20.953229 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.070387 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64abc33a14e24861b5edff29b8aeeb7d', 'request_id': 'req-a7066460-e371-4602-9e74-0017650c95cf'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:21.076191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64abc33a14e24861b5edff29b8aeeb7d', 'request_id': 'req-a7066460-e371-4602-9e74-0017650c95cf'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:21.082363 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64abc33a14e24861b5edff29b8aeeb7d', 'request_id': 'req-a7066460-e371-4602-9e74-0017650c95cf'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:21.171247 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] The token cache is being invalidate because project 64abc33a14e24861b5edff29b8aeeb7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:21.171470 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:21.172136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-a7066460-e371-4602-9e74-0017650c95cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:21.173719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1425/2839] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:20 2022] DELETE /identity/v3/projects/64abc33a14e24861b5edff29b8aeeb7d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:21.179395 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.179885 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.179984 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.197919 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:21.199114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:21.257351 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1415/2840] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:21.266153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.266451 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.266745 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.286319 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:21.290858 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:21.295927 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1426/2841] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:21.300069 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.310001 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199813-b4fc-4b41-872c-2b6641d752f4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.314701 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.315349 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.315632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.315908 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.316659 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:40:21.316971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.318593 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.330457 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1416/2842] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:40:21.335014 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.352827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303064b-1a58-4f85-bb94-91c35ac6d016 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.352827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.352827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.352827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.353239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.353239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.353915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.453596 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1427/2843] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:40:21 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:21.457847 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.468009 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab650e7-637e-444f-bf9b-8f19bfa8fd5d admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.525377 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.526040 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.526320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.526588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] PATH_INFO: `/v3/users` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.527396 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.529376 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.537198 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:21.623442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1417/2844] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:40:21 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:40:21.627647 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.638880 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082c322-7623-41d1-9610-0af0c97db5d3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.643475 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.644123 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.644403 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.644733 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.645457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.647557 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.657879 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1428/2845] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:21.663269 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.673556 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8def6d9-d9a8-4548-a99b-71ebdb2cb691 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.678768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.678768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.679014 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.679287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.679999 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.681595 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715ea54-e80b-46a9-bca5-5c00bafbd38c admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.692089 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1418/2846] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:40:21.696750 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.707482 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db37e6c-6815-4d25-b8a6-4a3a2e5d4452 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:21.712218 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:21.713228 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.713529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.713807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] PATH_INFO: `/v3/projects/1d68ad02d9924a56bbf5685cbca99047/users/847a5b09e1b744d28e1775046b94b04c/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.729344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d68ad02d9924a56bbf5685cbca99047, user_id=847a5b09e1b744d28e1775046b94b04c, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:21.732053 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:21.735019 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:21.818534 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1429/2847] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:40:21 2022] PUT /identity/v3/projects/1d68ad02d9924a56bbf5685cbca99047/users/847a5b09e1b744d28e1775046b94b04c/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:21.829081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:21.829374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:21.829648 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:21.859189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] MFA Rules not processed for user `847a5b09e1b744d28e1775046b94b04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:21.860386 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:21.927588 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1419/2848] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:21.938922 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:21.940765 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:22.001662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cbfc6-ec23-45f0-86bd-afe42736d791 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:22.006227 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:22.006923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:22.007213 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:22.007492 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:22.016391 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:22.017877 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde231e-8e39-486c-b491-bdeaeb0387e0 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:22.027487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1430/2849] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:31.206419 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:31.207992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:31.287923 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352ce10-41b6-4a08-98eb-108484b780b5 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:31.293702 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:31.294382 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:31.294650 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:31.294928 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:31.296880 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:31.315151 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:31.316757 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:31.328611 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1420/2850] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:32.023431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=28 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:41.323145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:INSERT=2,keystone:SELECT=73 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:43.185035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:43.185565 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:43.185673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:43.203209 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] MFA Rules not processed for user `a446f2a301a04fa485094d7be5219a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:43.204519 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:43.261127 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1431/2851] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:43.583096 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:43.584562 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:43.648456 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152438ed-f9d3-4b5c-b12e-174836d555b3 service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:43.652147 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:43.652828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:43.653114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:43.653389 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:43.655167 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:43.665954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:43.667374 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6aab3-3833-4df0-8ffa-f98f86c53d0b service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:43.676052 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1421/2852] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:49.138592 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:49.211153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5895f0-619b-4a34-bb40-79d2d450e9b3 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:49.215719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:49.216609 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:49.216950 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:49.217232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] PATH_INFO: `/v3/users/847a5b09e1b744d28e1775046b94b04c` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:49.218426 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847a5b09e1b744d28e1775046b94b04c)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:49.220529 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:49.377606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847a5b09e1b744d28e1775046b94b04c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:49.388830 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847a5b09e1b744d28e1775046b94b04c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:49.394801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847a5b09e1b744d28e1775046b94b04c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:49.462723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847a5b09e1b744d28e1775046b94b04c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:49.463397 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847a5b09e1b744d28e1775046b94b04c'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:49.482491 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1432/2853] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:40:49 2022] DELETE /identity/v3/users/847a5b09e1b744d28e1775046b94b04c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:49.487057 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:49.487637 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:49.491424 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:49.505913 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:40:49.507158 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:49.569667 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1422/2854] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:40:49.579985 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:49.581512 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:49.610369 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:40:49.641071 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f21c2-acb9-49cf-b4ac-57f7a8902020 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:49.645098 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:49.645740 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:49.646044 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:49.646323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:49.655499 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:49.656958 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:49.666820 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1433/2855] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:40:50.095137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:40:50.095137 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:40:50.107442 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e462d-6ebe-4519-930e-2afb5af8c755 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:40:50.115704 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:40:50.116726 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:40:50.118406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:40:50.118406 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] PATH_INFO: `/v3/projects/1d68ad02d9924a56bbf5685cbca99047` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:40:50.118591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d68ad02d9924a56bbf5685cbca99047)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:40:50.120534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:40:50.243423 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d68ad02d9924a56bbf5685cbca99047', 'request_id': 'req-98162f61-0da4-4b4a-a5f3-2be833145d4a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:50.249673 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d68ad02d9924a56bbf5685cbca99047', 'request_id': 'req-98162f61-0da4-4b4a-a5f3-2be833145d4a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:50.255134 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d68ad02d9924a56bbf5685cbca99047', 'request_id': 'req-98162f61-0da4-4b4a-a5f3-2be833145d4a'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:50.331542 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] The token cache is being invalidate because project 1d68ad02d9924a56bbf5685cbca99047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:40:50.331832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:50.332473 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:40:50.334066 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1423/2856] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:40:50 2022] DELETE /identity/v3/projects/1d68ad02d9924a56bbf5685cbca99047 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:40:59.697748 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=15 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:41:00.328242 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:41:02.068239 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:41:02.069714 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:41:02.081065 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4a888-08f7-4d44-9d8d-695bf3a47f9d service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:41:02.130038 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:41:02.130678 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:41:02.130963 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:41:02.131818 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:41:02.133618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:41:02.195838 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:41:02.197326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3307de-8e18-47db-b7c6-bb6c3becb10e service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:41:02.206894 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1434/2857] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:41:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:41:12.202631 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=50 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:41:40.302471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:41:40.303904 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:41:40.394090 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162f61-0da4-4b4a-a5f3-2be833145d4a admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:41:40.399712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:41:40.400352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:41:40.400665 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:41:40.400943 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:41:40.402703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:41:40.414078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:41:40.415769 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:41:40.425850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1424/2858] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:41:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:41:50.421531 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=41 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:41:54.521276 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:41:54.521778 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:41:54.521885 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:41:54.538395 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] MFA Rules not processed for user `2e99a37e30d547b992b4ede147f684a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:41:54.539597 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:41:54.595807 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1435/2859] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:41:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:42:04.589018 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=25 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:05.573678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:05.585282 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad2b-1dca-4ace-8092-810d5514e606 service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:05.591536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:05.591536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:05.591536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:05.591536 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:05.593204 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:05.605268 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:05.606845 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:05.616907 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1425/2860] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:43:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:43:05.684230 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:05.685951 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:05.752396 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c841fe2-34fc-4646-95a8-a3fe2f47557a None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:05.756948 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:05.757815 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:05.758101 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:05.758379 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] PATH_INFO: `/v3/users/2e99a37e30d547b992b4ede147f684a4` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:05.759571 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e99a37e30d547b992b4ede147f684a4)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:05.761552 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:05.942068 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e99a37e30d547b992b4ede147f684a4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:05.948687 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e99a37e30d547b992b4ede147f684a4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:05.949342 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e99a37e30d547b992b4ede147f684a4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.009457 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e99a37e30d547b992b4ede147f684a4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.010089 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-19c8742b-ed13-462c-ac95-66007266bb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e99a37e30d547b992b4ede147f684a4'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.028759 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1436/2861] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:43:05 2022] DELETE /identity/v3/users/2e99a37e30d547b992b4ede147f684a4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:43:06.245370 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.256038 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971111-06fa-4779-a758-4dc1f53eab46 service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.323354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.324189 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.324471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.324775 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] PATH_INFO: `/v3/projects/a0a185c667b14f19aee6427e4f91af8d` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.325955 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a185c667b14f19aee6427e4f91af8d)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.327848 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.420607 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a185c667b14f19aee6427e4f91af8d', 'request_id': 'req-78f93854-0a04-4750-8d38-11d048c8dc28'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.426929 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a185c667b14f19aee6427e4f91af8d', 'request_id': 'req-78f93854-0a04-4750-8d38-11d048c8dc28'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.432471 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a185c667b14f19aee6427e4f91af8d', 'request_id': 'req-78f93854-0a04-4750-8d38-11d048c8dc28'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.488422 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] The token cache is being invalidate because project a0a185c667b14f19aee6427e4f91af8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:43:06.488674 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.489292 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-78f93854-0a04-4750-8d38-11d048c8dc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:43:06.490852 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1426/2862] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:43:06 2022] DELETE /identity/v3/projects/a0a185c667b14f19aee6427e4f91af8d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:43:06.495981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.496275 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.496561 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.524967 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:06.526176 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:06.583656 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1437/2863] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:06.591343 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.591819 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.591949 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.610077 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:06.611250 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:06.618932 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1427/2864] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:06.623035 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.633129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf6672-928f-4e1c-b14a-b582d612eab2 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.637662 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.638298 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.638578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.638850 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.639627 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80907) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:43:06.639938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.642162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.648718 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:06.653789 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1438/2865] 10.222.0.57 () {58 vars in 1152 bytes} [Tue Jun 7 09:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:43:06.657946 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.666920 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668522e-b9f7-4fa3-8e94-0a3c28d963b4 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.671547 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.672197 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.672990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.673267 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.674870 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.676709 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.747930 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1428/2866] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:43:06 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:43:06.752431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.762835 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc8218-8c98-4abc-8d66-c7e9bf05eda3 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.824717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.825380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.825670 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.825945 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.826636 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.828546 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.911725 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1439/2867] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:43:06 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:43:06.915771 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.925899 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc7ef-8d44-42b5-83a1-5bee2018ea65 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.930432 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.931940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.931940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.931940 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.932303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.933914 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.944117 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1429/2868] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:43:06.947966 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.958618 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0111f5-7d25-4475-9e61-2ae1274b1906 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.963308 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.963959 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.964244 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.964524 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:06.965283 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:06.966878 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b152f-5b80-49bf-8455-f9cbb40c1073 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:06.977616 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1440/2869] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:43:06.981587 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:06.991357 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693ee4a-c39b-40fa-a9fb-8d4fc7cf3849 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:06.995864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:06.996789 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:06.997064 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:06.997333 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] PATH_INFO: `/v3/projects/17f7583c3efc45cb83a18d3c8f3f1cf6/users/4658a11ae5b740e78c4ea8e656b46212/roles/da277faf5b1141e1aaa9bdaf1713ae18` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:07.012990 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f7583c3efc45cb83a18d3c8f3f1cf6, user_id=4658a11ae5b740e78c4ea8e656b46212, role_id=da277faf5b1141e1aaa9bdaf1713ae18)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:07.015790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:07.084521 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1430/2870] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:43:06 2022] PUT /identity/v3/projects/17f7583c3efc45cb83a18d3c8f3f1cf6/users/4658a11ae5b740e78c4ea8e656b46212/roles/da277faf5b1141e1aaa9bdaf1713ae18 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:43:07.089578 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436c2a-3e8b-451f-998d-b2cf758f0574 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:07.089871 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436c2a-3e8b-451f-998d-b2cf758f0574 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:07.090153 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436c2a-3e8b-451f-998d-b2cf758f0574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:07.108148 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-03436c2a-3e8b-451f-998d-b2cf758f0574 None None] MFA Rules not processed for user `a90171f1014d4a6dbe696da11bc1dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:07.109400 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-03436c2a-3e8b-451f-998d-b2cf758f0574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:07.168162 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1441/2871] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:43:07.177777 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:07.179545 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:07.194320 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:07.237113 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966cb1-119e-4063-8cde-3bf1b0d05518 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:07.240851 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:07.241488 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:07.241770 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:07.242045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:07.251480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:07.252922 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:07.262286 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1431/2872] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:43:12.436207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:12.436780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:12.436888 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:12.465876 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] MFA Rules not processed for user `4658a11ae5b740e78c4ea8e656b46212`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:12.469358 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:12.568731 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1442/2873] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:12.571632 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:12.581719 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8e89-7d90-473b-ab5b-dff35fcad334 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:12.586371 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:12.587023 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:12.587303 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:12.587574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] PATH_INFO: `/v3/projects` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:12.588258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:12.590184 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:12.676401 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1432/2874] 10.222.0.57 () {60 vars in 1152 bytes} [Tue Jun 7 09:43:12 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:43:12.679095 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:12.691326 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb69e46-ebb0-4d31-b2fc-a5c5a51e1c45 None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:12.695548 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:12.753756 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:12.754775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:12.754775 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:12.755105 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:12.755813 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:12.757802 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:12.856310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1443/2875] 10.222.0.57 () {60 vars in 1143 bytes} [Tue Jun 7 09:43:12 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:43:12.861102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:12.872468 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f530b-cb02-4973-8edd-f37276e82aed admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:12.877646 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:12.878305 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:12.878591 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:12.878864 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:12.879597 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:12.881244 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:12.893054 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1433/2876] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:43:12.897522 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:12.908556 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c2e4f-0747-44c5-a553-85993cf907f9 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:12.921287 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:12.922884 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:12.923185 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:12.923495 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] PATH_INFO: `/v3/projects/d84c4d50c7044dad9bb62d0182963d76/users/7372233af3bc49f598cb205a0e135130/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:12.944912 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d84c4d50c7044dad9bb62d0182963d76, user_id=7372233af3bc49f598cb205a0e135130, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:12.947606 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:13.019652 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1444/2877] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:43:12 2022] PUT /identity/v3/projects/d84c4d50c7044dad9bb62d0182963d76/users/7372233af3bc49f598cb205a0e135130/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:43:13.020903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:13.032078 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f67bb-4d22-4a13-841f-16171f4757e1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:13.093560 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:13.094786 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:13.095352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:13.095631 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:13.096359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:13.097994 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46186d0a-d9f5-4b66-9784-93e406743f56 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:13.108711 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1434/2878] 10.222.0.57 () {58 vars in 1121 bytes} [Tue Jun 7 09:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:43:13.113030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:13.124030 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6dbe-9d65-413e-9b83-54cb7f34203f admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:13.128848 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:13.129734 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:13.130020 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:13.130295 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] PATH_INFO: `/v3/projects/d84c4d50c7044dad9bb62d0182963d76/users/7372233af3bc49f598cb205a0e135130/roles/8c0f69603d9b44b0b45e7b500a8f109a` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:13.132285 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d84c4d50c7044dad9bb62d0182963d76, user_id=7372233af3bc49f598cb205a0e135130, role_id=8c0f69603d9b44b0b45e7b500a8f109a)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:13.135048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:13.146702 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1445/2879] 10.222.0.57 () {60 vars in 1482 bytes} [Tue Jun 7 09:43:13 2022] PUT /identity/v3/projects/d84c4d50c7044dad9bb62d0182963d76/users/7372233af3bc49f598cb205a0e135130/roles/8c0f69603d9b44b0b45e7b500a8f109a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:43:18.522118 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df7ee-fba8-4cc4-81a7-2c88adfc2013 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:18.522640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df7ee-fba8-4cc4-81a7-2c88adfc2013 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:18.522640 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df7ee-fba8-4cc4-81a7-2c88adfc2013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:18.549867 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3f2df7ee-fba8-4cc4-81a7-2c88adfc2013 None None] MFA Rules not processed for user `7372233af3bc49f598cb205a0e135130`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:18.551159 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3f2df7ee-fba8-4cc4-81a7-2c88adfc2013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:18.620534 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1435/2880] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:43:18.635032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:18.637201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:18.728274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0900-d303-4492-90d1-557624539a29 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:18.732033 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:18.732713 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:18.733003 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:18.733305 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:18.789310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:18.793544 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:18.795002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:18.805119 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1446/2881] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:43:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:43:19.072539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eecb0c-0e16-4270-9832-8a99e98b99a6 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:19.074744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eecb0c-0e16-4270-9832-8a99e98b99a6 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:19.075035 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eecb0c-0e16-4270-9832-8a99e98b99a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:19.080876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:19.139307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f4eecb0c-0e16-4270-9832-8a99e98b99a6 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:19.140641 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f4eecb0c-0e16-4270-9832-8a99e98b99a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:19.220274 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1436/2882] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:19.402530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:19.404051 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:19.499205 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04cf-24a4-479b-826c-48cdf256a035 service glance] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:19.504772 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:19.505417 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:19.505697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:19.505970 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:19.514923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:19.517164 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:19.525752 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1447/2883] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:43:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:43:29.307120 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=45 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:29.522191 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=44 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:30.291133 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:30.291428 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:30.291712 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:30.309974 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] MFA Rules not processed for user `4658a11ae5b740e78c4ea8e656b46212`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:30.311176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:30.318263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1437/2884] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:30.354383 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:30.364002 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43991c72-9aff-466f-9148-bb7cc0d06cae service swift] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:30.367694 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7025724d5c31460db223411aab77003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:30.368349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:30.368661 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:30.368940 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:30.370719 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:30.453852 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:30.455294 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57be52-0d72-4d1b-8fb0-072406482ad3 service glance] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:30.476278 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1448/2885] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:43:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:43:31.348152 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:31.349584 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:31.410890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7b3c-e783-45c2-b048-ca78b6c9aed3 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:31.414546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:31.415194 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:31.415480 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:31.415760 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:31.417606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:31.431675 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:31.433153 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:31.445169 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1438/2886] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:43:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:43:34.111013 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2fcd7-f40b-4dbf-a5c1-7327aa057e07 None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:34.111564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2fcd7-f40b-4dbf-a5c1-7327aa057e07 None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:34.111564 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2fcd7-f40b-4dbf-a5c1-7327aa057e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:34.129642 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-d6a2fcd7-f40b-4dbf-a5c1-7327aa057e07 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:34.130875 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d6a2fcd7-f40b-4dbf-a5c1-7327aa057e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:34.139804 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1449/2887] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Jun 7 09:43:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:43:34.152898 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:43:34.166245 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8966-872b-4d39-8993-4a936a61d018 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:43:34.172307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:43:34.173000 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:34.173294 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:34.173574 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:34.183131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:43:34.184768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:43:34.194644 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1439/2888] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 09:43:44.123981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=46 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:44.190262 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=35 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:43:53.917643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] REQUEST_METHOD: `POST` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:43:53.918143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:43:53.918254 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:43:53.936076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.auth.core [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] MFA Rules not processed for user `a446f2a301a04fa485094d7be5219a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80907) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:43:53.937310 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:43:53.993343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1450/2889] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:43:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:44:03.985808 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=25 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:44:08.116263 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:44:08.125575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df07-ff9a-4237-a444-986a96eb0c5a service swift] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:44:08.129114 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:44:08.129745 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:44:08.130026 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:44:08.130300 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:44:08.139206 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:44:08.140818 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:44:08.150399 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1440/2890] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:44:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:44:15.105349 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:44:15.106861 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:44:15.122827 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fb8af-d27a-459e-978c-4dcaddfe65ec None None] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:44:15.128355 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:44:15.129045 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:44:15.129332 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:44:15.129608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:44:15.131870 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:44:15.144179 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:44:15.145801 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:44:15.157197 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1451/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:44:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:44:18.146699 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=6 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:44:25.152599 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:45:02.507031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:45:02.516258 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02776102-c6eb-464f-9f38-7c87b14af478 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:45:02.519906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:45:02.520549 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:45:02.520871 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:45:02.521138 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:45:02.522884 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:45:02.533678 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:45:02.537816 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d614bb6-e31a-4bfb-9512-a2abc33d4b7a service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:45:02.543754 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1441/2892] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:45:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:45:12.539810 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=6 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:03.452608 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:03.454032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:03.529957 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559348b-a3a8-4699-bec3-9a73e58cd014 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:03.534061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d851c1cb8b495a87692b8e8970f851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:03.534692 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:03.534972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:03.535248 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:03.537026 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:03.547545 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:03.549598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:03.558833 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1452/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:13.554408 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=35 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:15.136656 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:15.137255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:15.137255 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:15.154321 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] MFA Rules not processed for user `a446f2a301a04fa485094d7be5219a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:46:15.155948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:15.162910 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1442/2894] 10.222.0.57 () {58 vars in 957 bytes} [Tue Jun 7 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:46:15.470337 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:15.480201 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c68ab1-68a0-4829-ad86-259138d8bea2 service cinder] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:15.483791 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:15.484422 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:15.484747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:15.485029 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:15.486821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:15.497132 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:15.498598 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:15.506773 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1453/2895] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:16.385463 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:16.393148 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:16.471496 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b4cd8-a482-405c-8404-c93df474e27d None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:16.476509 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:16.477173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:16.477478 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:16.477727 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:16.479514 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:16.491297 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:16.492903 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b61bbf-2297-4342-86e6-9bbdf2a5addf service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:16.508145 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1443/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:20.702011 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:20.705152 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:20.732142 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b6596-e343-4260-9e75-dd79d1059b15 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:20.738728 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:20.739547 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:20.739832 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:20.740103 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:20.742032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:20.757059 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:20.758972 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:20.771479 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1454/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:26.498865 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=35 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:30.766873 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=12 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:31.589433 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] REQUEST_METHOD: `POST` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:31.589939 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:31.590046 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:31.608546 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.auth.core [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] MFA Rules not processed for user `37821819ba394afda7a92b7aa99096b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:46:31.609766 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:31.617505 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1444/2898] 10.222.0.57 () {58 vars in 967 bytes} [Tue Jun 7 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:46:31.656843 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:31.668708 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b801ea3-0d6f-49b5-8165-33c2cd6cb6a1 service nova] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:31.673946 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327a3409391e4d558aeab965401f48e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:31.675530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:31.675530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:31.675530 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:31.683737 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:31.685440 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:31.694331 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1455/2899] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Jun 7 09:46:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:46:31.760889 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:31.775466 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ca354-4a74-4194-b859-d6c948c79912 None None] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:31.782136 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dff6b630f6be464aa6cdcc73da587e4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:31.782830 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:31.783106 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:31.783383 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:31.785301 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:31.798170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:31.799744 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:31.811081 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1445/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:34.632915 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:34.642314 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5068c-d4eb-474f-a2eb-ec4ef578e3ed service swift] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:34.646048 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:34.646673 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:34.646954 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:34.647223 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:34.649052 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:34.718747 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:34.720246 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:34.731175 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1456/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:37.110280 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:37.111694 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:37.125966 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b20cb-54f6-4b70-b4ec-95d9a305a197 service nova] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:37.131065 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:37.131853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:37.132170 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:37.132450 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:37.134291 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:37.146548 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:37.148682 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:37.158828 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1446/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:41.115665 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:41.179572 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66de7-abbc-48e8-8dec-3213fe197951 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:41.184255 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:41.185155 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:41.185444 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:41.185723 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] PATH_INFO: `/v3/users/4658a11ae5b740e78c4ea8e656b46212` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:41.186902 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4658a11ae5b740e78c4ea8e656b46212)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:41.188855 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:41.373487 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4658a11ae5b740e78c4ea8e656b46212'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.380515 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4658a11ae5b740e78c4ea8e656b46212'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.381131 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4658a11ae5b740e78c4ea8e656b46212'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.457318 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4658a11ae5b740e78c4ea8e656b46212'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.457937 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4658a11ae5b740e78c4ea8e656b46212'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.472635 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:41.476924 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1457/2903] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:46:41 2022] DELETE /identity/v3/users/4658a11ae5b740e78c4ea8e656b46212 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:46:41.692998 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:41.706575 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876dc-e7d6-4000-bef1-760e9bda69fe service placement] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:41.764420 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:41.765302 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:41.765581 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:41.765857 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] PATH_INFO: `/v3/projects/17f7583c3efc45cb83a18d3c8f3f1cf6` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:41.767031 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f7583c3efc45cb83a18d3c8f3f1cf6)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:41.768954 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:41.857441 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f7583c3efc45cb83a18d3c8f3f1cf6', 'request_id': 'req-25a04811-558d-49fa-b829-e77d8ea075c1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.863628 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f7583c3efc45cb83a18d3c8f3f1cf6', 'request_id': 'req-25a04811-558d-49fa-b829-e77d8ea075c1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.869585 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f7583c3efc45cb83a18d3c8f3f1cf6', 'request_id': 'req-25a04811-558d-49fa-b829-e77d8ea075c1'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.966364 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] The token cache is being invalidate because project 17f7583c3efc45cb83a18d3c8f3f1cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:46:41.966649 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.967270 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:41.968892 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1447/2904] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:46:41 2022] DELETE /identity/v3/projects/17f7583c3efc45cb83a18d3c8f3f1cf6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:46:41.972938 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:41.974344 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:41.987183 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5ee4-7cfb-44b8-bb75-64a6a3916609 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:42.047685 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:42.048555 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:42.048869 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:42.049147 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] PATH_INFO: `/v3/users/7372233af3bc49f598cb205a0e135130` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:42.050317 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7372233af3bc49f598cb205a0e135130)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:42.052232 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:42.273688 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7372233af3bc49f598cb205a0e135130'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.280380 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7372233af3bc49f598cb205a0e135130'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.281061 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7372233af3bc49f598cb205a0e135130'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.346511 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7372233af3bc49f598cb205a0e135130'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.347136 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7372233af3bc49f598cb205a0e135130'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.365448 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1458/2905] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:46:41 2022] DELETE /identity/v3/users/7372233af3bc49f598cb205a0e135130 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:46:42.616141 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:42.617609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:42.630863 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04811-558d-49fa-b829-e77d8ea075c1 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:42.683179 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:42.688876 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:42.689718 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:42.689992 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:42.690259 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] PATH_INFO: `/v3/projects/d84c4d50c7044dad9bb62d0182963d76` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:42.691430 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d84c4d50c7044dad9bb62d0182963d76)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:42.693354 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:42.812539 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd84c4d50c7044dad9bb62d0182963d76', 'request_id': 'req-c829b5c0-b790-4e62-b9ff-c12b089f78ce'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.818853 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd84c4d50c7044dad9bb62d0182963d76', 'request_id': 'req-c829b5c0-b790-4e62-b9ff-c12b089f78ce'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.824935 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd84c4d50c7044dad9bb62d0182963d76', 'request_id': 'req-c829b5c0-b790-4e62-b9ff-c12b089f78ce'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.928890 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] The token cache is being invalidate because project d84c4d50c7044dad9bb62d0182963d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:46:42.929176 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.929790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:46:42.931359 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1448/2906] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:46:42 2022] DELETE /identity/v3/projects/d84c4d50c7044dad9bb62d0182963d76 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:46:46.781361 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:46:46.783032 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:46.986517 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c0049-e08f-41e4-8b66-ff057ed56acf admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:46:47.004368 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:47.046274 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:46:47.046923 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:46:47.047207 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:46:47.047484 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:46:47.049423 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:46:47.061076 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:46:47.062645 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:46:47.073292 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1459/2907] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:46:52.925654 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:46:57.102814 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=22 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:48:30.133435 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:30.134906 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:30.192573 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b5c0-b790-4e62-b9ff-c12b089f78ce admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:30.196241 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:30.196859 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:30.197144 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:30.197421 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:30.199214 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:30.274142 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:30.275465 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:30.294724 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1449/2908] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:48:31.456008 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:31.457437 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:31.469481 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a95e-5c21-46bf-87f7-901e93df2a89 service placement] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:31.473073 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:31.473697 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:31.473977 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:31.474245 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:31.475968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:31.489243 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:31.490660 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:31.501497 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1460/2909] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:48:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:48:40.288668 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=54 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:48:41.496981 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=6 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:48:43.149671 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:43.158794 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0d76-b690-4ea7-8593-b4dfb2c58cb8 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:43.162352 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943602b745284e6b9a53ff217a9ee0b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:43.162976 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:43.163254 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:43.163525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:43.165287 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:43.227768 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:43.229239 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:43.239045 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1450/2910] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:48:49.898821 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:49.900323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:49.914727 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e6875-4a5a-495b-9fdb-5437e3226626 service neutron] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:49.919968 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:49.920637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] REQUEST_METHOD: `GET` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:49.920926 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:49.921202 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:49.922975 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:49.935210 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:49.936883 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:49.947420 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1461/2911] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:48:50.036868 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:50.038323 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:50.053135 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242bcc7-7926-496d-b002-37bc305fc308 service neutron] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:50.058102 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:50.059544 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:50.059832 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:50.060230 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] PATH_INFO: `/v3/users/a446f2a301a04fa485094d7be5219a07` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:50.064951 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a446f2a301a04fa485094d7be5219a07)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:50.066947 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:50.406275 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a446f2a301a04fa485094d7be5219a07'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.412936 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a446f2a301a04fa485094d7be5219a07'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.497971 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a446f2a301a04fa485094d7be5219a07'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.498606 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a446f2a301a04fa485094d7be5219a07'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.499212 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a446f2a301a04fa485094d7be5219a07'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.516751 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1451/2912] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:48:50 2022] DELETE /identity/v3/users/a446f2a301a04fa485094d7be5219a07 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:48:50.768762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:50.780279 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9fbc-c6cf-47e3-9dfb-9d2b0b8b21c8 service placement] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:50.835268 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:50.836122 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:50.836405 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:50.836717 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] PATH_INFO: `/v3/projects/8c26e375cbb2421b8c8e34653f7d2f38` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:50.838144 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c26e375cbb2421b8c8e34653f7d2f38)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:50.840762 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:50.948498 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c26e375cbb2421b8c8e34653f7d2f38', 'request_id': 'req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.954489 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c26e375cbb2421b8c8e34653f7d2f38', 'request_id': 'req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:50.960528 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c26e375cbb2421b8c8e34653f7d2f38', 'request_id': 'req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.037726 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] The token cache is being invalidate because project 8c26e375cbb2421b8c8e34653f7d2f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:48:51.037974 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.038643 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.040143 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1462/2913] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:48:50 2022] DELETE /identity/v3/projects/8c26e375cbb2421b8c8e34653f7d2f38 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:48:51.044131 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:51.045576 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:51.059512 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe3f14-1272-4453-b9eb-88e7bcd6b016 admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:51.117086 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:51.117948 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:51.118225 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:51.118494 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] PATH_INFO: `/v3/users/b669c71e4d1143b9a9c3259dfd5850be` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:51.119660 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b669c71e4d1143b9a9c3259dfd5850be)` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:51.121609 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:51.126748 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:48:51.291271 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b669c71e4d1143b9a9c3259dfd5850be'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.298084 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b669c71e4d1143b9a9c3259dfd5850be'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.398173 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b669c71e4d1143b9a9c3259dfd5850be'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.398792 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b669c71e4d1143b9a9c3259dfd5850be'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.399409 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.notifications [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b669c71e4d1143b9a9c3259dfd5850be'} {{(pid=80906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.417082 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1452/2914] 10.222.0.57 () {58 vars in 1223 bytes} [Tue Jun 7 09:48:51 2022] DELETE /identity/v3/users/b669c71e4d1143b9a9c3259dfd5850be => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:48:51.680551 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Authenticating user token {{(pid=80907) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:48:51.682075 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.fernet_utils [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80907) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:48:51.696343 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b75dab-874c-479f-a1b2-feee3a8ecfe0 admin admin] Validating token access rules against request {{(pid=80907) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:48:51.757780 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a90171f1014d4a6dbe696da11bc1dd7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '047614873c7a462ea30132f8c49e5842', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80907) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:48:51.758637 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:48:51.758921 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:48:51.759198 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] PATH_INFO: `/v3/projects/9ab7584c6e1b49248b951072bef46503` {{(pid=80907) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:48:51.760374 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab7584c6e1b49248b951072bef46503)` {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:48:51.762323 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] RBAC: Authorization granted {{(pid=80907) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:48:51.782792 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:48:51.889859 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab7584c6e1b49248b951072bef46503', 'request_id': 'req-c607a91a-7fb1-4b93-96a5-e2454e78f317'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.895190 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab7584c6e1b49248b951072bef46503', 'request_id': 'req-c607a91a-7fb1-4b93-96a5-e2454e78f317'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.901169 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab7584c6e1b49248b951072bef46503', 'request_id': 'req-c607a91a-7fb1-4b93-96a5-e2454e78f317'} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.963521 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] The token cache is being invalidate because project 9ab7584c6e1b49248b951072bef46503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80907) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:48:51.964129 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.964377 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG keystone.notifications [None req-c607a91a-7fb1-4b93-96a5-e2454e78f317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80907) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:48:51.968269 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: [pid: 80907|app: 0|req: 1463/2915] 10.222.0.57 () {58 vars in 1232 bytes} [Tue Jun 7 09:48:51 2022] DELETE /identity/v3/projects/9ab7584c6e1b49248b951072bef46503 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:49:01.413583 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:49:02.069431 dv-cinder-836996-31-iscsi devstack@keystone.service[80907]: DEBUG dbcounter [-] [80907] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80907) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:51:50.188634 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Authenticating user token {{(pid=80906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:51:50.190095 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:51:50.327860 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e794d-4864-4043-967d-43a38ab552db admin admin] Validating token access rules against request {{(pid=80906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:51:50.386788 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6459963216fe4a039289498f1fd8af54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05821a6472b4ed48bf91eaae8471b52', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:51:50.387426 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] REQUEST_METHOD: `GET` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:51:50.387703 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] SCRIPT_NAME: `/identity` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:51:50.388003 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:51:50.389790 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.fernet_utils [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:51:50.401307 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:51:50.402850 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0933ba-1025-4299-b83b-9408f7e9efcb service placement] RBAC: Authorization granted {{(pid=80906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:51:50.412525 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: [pid: 80906|app: 0|req: 1453/2916] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Jun 7 09:51:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:52:00.408452 dv-cinder-836996-31-iscsi devstack@keystone.service[80906]: DEBUG dbcounter [-] [80906] Writing DB stats keystone:SELECT=89 {{(pid=80906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}